Spaces:

John6666

/

votepurchase-multiple-model

like 65

Running on Zero

PAG is possible. Or rather, stablepy supports it, but I just haven't made a GUI for it. I'm just being lazy.😆
I wonder what the rest of it is like...?
For better or worse, it's abstracted using stablepy, so it'll probably break if it's not supported by stablepy.
DiffuseCraft is a demo for stablepy, so most of the things that stablepy itself can do have been implemented. So, if it's a feature that DiffuseCraft has, I can add it straight away. If it's a feature that DiffuseCraft doesn't have, but is supported by the pip version of Diffusers, r3gm will probably support it quite quickly, like he did with the scheduler the other day.
He is a busy but friendly and proactive person, so he will probably answer your questions unless they are unreasonable requests in a programmatic context.
If it's something really simple, I can even submit a PR.
https://huggingface.co/spaces/r3gm/DiffuseCraft/discussions?status=open&type=discussion
https://github.com/R3gm/stablepy

Edit:
Since we're here, let's identify the features that DiffuseCraft (stablepy) is missing. It would be easier if we put together a list of ideas. He writes 100 times faster and more accurately than I do...
This is purely a difference in coding ability...

Edit:
Also, tell me which features you want to see prioritized that are not yet in VP, but are in DiffuseCraft. I'll add PAG first.

I added PAG scale and FreeU.

I added PAG scale and FreeU.

Thanks!

Since we're here, let's identify the features that DiffuseCraft (stablepy) is missing. It would be easier if we put together a list of ideas. He writes 100 times faster and more accurately than I do...
This is purely a difference in coding ability...

There is a kind of scheduler/samplers that are very promising, I'd love to see them supported in stablepy. They're called CFG++ or cfgpp in comfyui, https://arxiv.org/abs/2406.08070
Aside from this I think stablepy is feature complete till now.

I also believe that img2img in VP (supported by diffuse craft) would be very good, controlnets and IPAdapters would be great too !

Also inference with lycoris!
I can't use LoKrs or LoHAs on VP or DiffuseCraft. there are alot of LyCORISs out there that are better than LoRAs but I'm unable to use them since they're not supported in stablepy (neither are they in diffusers) so this'll probably be a great addition

Thanks.
I've also been wondering about LyCORIS, and I can use it if I call it directly from PEFT, but as you say, there is no way to call it from Diffusers. I think r3gm has the ability to create functions that are not in Diffusers, but I think the wrapping of Diffusers is the theme of stablepy, so it would be better to improve Diffusers itself first...
The issue with the LyCORIS implementation is how to determine that it is a LyCORIS file. I think this is also true for other LoRA variants.

Is CFG++ supported by Diffusers?

Edit:
Oh... its similar rescale.

Typically, LyCoris algorithms/models contain an identifier called "hada" in the keys which I believe stands for "hadamard product"

https://github.com/KohakuBlueleaf/LyCORIS/blob/main/docs/Algo-Details.md

This is also discovered here by sayakpaul
https://github.com/huggingface/diffusers/issues/4133

There was also an issue about this here https://github.com/huggingface/diffusers/issues/3087

I also found this gist about lycoris inference
https://gist.github.com/adhikjoshi/2c6da89cbcd7a6a3344d3081ccd1dda0

Is CFG++ supported by Diffusers?

I believe not.

Tho reForge webui implemented it
https://github.com/Panchovix/stable-diffusion-webui-reForge

https://cfgpp-diffusion.github.io/

I have informed r3gm about guidance_rescale. Good night.😪

This is also discovered here by sayakpaul

There may be some issues with sayakpaul knowing about it but not having it implemented...
It looks like it still hasn't been implemented even with this major LoRA renovation.

Edit:
PEFT's one.
https://huggingface.co/docs/peft/package_reference/adapter_utils

Edit:
CFG++ is not easy even in Forge (nor reForge)?
https://github.com/lllyasviel/stable-diffusion-webui-forge/discussions/1864

CFG++ is not easy even in Forge (nor reForge)

CFG++ is implemented in reforge.
https://github.com/Panchovix/stable-diffusion-webui-reForge/blob/main/modules%2Fsd_samplers_extra.py#L77

https://github.com/Panchovix/stable-diffusion-webui-reForge/blob/main/ldm_patched%2Fk_diffusion%2Fsampling.py#L2167

I have informed r3gm about guidance_rescale.

Great!

There may be some issues with sayakpaul knowing about it but not having it implemented...

I thought so too.

It looks like it still hasn't been implemented even with this major LoRA renovation.

I'm also unsure why it hasn't been implemented

CFG++ is implemented in reforge.

yea.

@xi0v @r3gm
https://github.com/huggingface/diffusers/issues/4133#issuecomment-2510141183
I was told that LyCORIS should already be supported. I'm sure there is some LyCORIS that will cause changes when applied, but I can't seem to find any logic that reads like a header...I wonder if PEFT is taking over that part of the process.
It would be nice if there was a LyCORIS that would produce the problem.

Hmmm

I suggest we try to find a LoHA Or a LoKr.

I believe the way to distinguish between a Lora and a LoHA should be the hadaidentifier in the lyco keys.
Otherwise, if diffusers has implemented a way to automatically load lycoris as if you were loading a regular lora, it should work as is.

Taken from https://github.com/KohakuBlueleaf/LyCORIS/blob/main/lycoris%2Fmodules%2Floha.py#L10-L29

class LohaModule(LycorisBaseModule):
    name = "loha"
    support_module = {
        "linear",
        "conv1d",
        "conv2d",
        "conv3d",
    }
    weight_list = [
        "hada_w1_a",
        "hada_w1_b",
        "hada_w2_a",
        "hada_w2_b",
        "hada_t1",
        "hada_t2",
        "alpha",
        "dora_scale",
    ]
    weight_list_det = ["hada_w1_a"]

as for LoKr, the identifier should be lokr

Taken from https://github.com/KohakuBlueleaf/LyCORIS/blob/main/lycoris%2Fmodules%2Flokr.py#L23-L43

class LokrModule(LycorisBaseModule):
    name = "kron"
    support_module = {
        "linear",
        "conv1d",
        "conv2d",
        "conv3d",
    }
    weight_list = [
        "lokr_w1",
        "lokr_w1_a",
        "lokr_w1_b",
        "lokr_w2",
        "lokr_w2_a",
        "lokr_w2_b",
        "lokr_t1",
        "lokr_t2",
        "alpha",
        "dora_scale",
    ]
    weight_list_det = ["lokr_w1", "lokr_w1_a"]

So a hacky way would be to dump lyco keys each time a lyco is being loaded and trying to find the identifier, tho this is definitely not the best way ever.

I see. There are some differences even within LyCORIS? If I can find one or two LyCORIS files that don't actually work, I can raise an issue, so I'll try to find some later. If they all do work, then that solves the problem.

I see. There are some differences even within LyCORIS

Yup.

If I can find one or two LyCORIS files that don't actually work

Time to browse civit 😅

I'm currently working on improving the pipeline code, so I'll do that first. Before that, I still have to finish my daily routine...

Hopefully I'm not out of place dropping in on this, but I have a Flux LoKr you could test out here: https://huggingface.co/davidrd123/Mary-Cassatt-Oil-FullAndCrops-Phase-4-beta_2-4_ss0_7-Flux-LoKr

Thank you! You're not wrong. However, as I understand it at the moment, LyCORIS requires assistance from external libraries in order to operate fully. It is possible that this is because it is LoKr.
Incidentally, the error that occurs when you perform inference using the GUI there is another unresolved bug in HF.😅

from diffusers import DiffusionPipeline
from lycoris import create_lycoris_from_weights

Edit:
I spent too much time on the pipeline and couldn't get to LyCORIS today, so I'll try writing the code tomorrow...

Can we get IPAdapters (and maybe controlnets) in VP? @John6666
I believe diffusecraft supports both.

With the ability to download custom ones, like the NoobAi ones