Convert to diffusers
#3
by
patrickvonplaten
- opened
Amazing work! Would you be interested to also provide a diffusers version of this checkpoint:
https://huggingface.co/docs/diffusers/main/en/api/pipelines/stable_diffusion/controlnet
?
I'd be happy to help with the conversion: https://github.com/huggingface/diffusers/blob/main/scripts/convert_original_controlnet_to_diffusers.py
Thanks! Yes we can try to help with this on Monday. Any help you can provide would be great.
(onnx.venv) N:\StableDiffusionUI-Shark-AMD\data\repo\diffusion_scripts\modules\cn_face>py conv.py --checkpoint_path=".\controlnet_sd21_laion_face_v2_pruned.pth" --dump_path="./test/" --original_config_file="cfg.yaml"
Traceback (most recent call last):
File "N:\StableDiffusionUI-Shark-AMD\data\repo\diffusion_scripts\modules\cn_face\conv.py", line 80, in <module>
controlnet = download_controlnet_from_original_ckpt(
File "N:\StableDiffusionUI-Shark-AMD\data\repo\onnx.venv\lib\site-packages\diffusers\pipelines\stable_diffusion\convert_from_ckpt.py", line 1354, in download_controlnet_from_original_ckpt
controlnet_model = convert_controlnet_checkpoint(
File "N:\StableDiffusionUI-Shark-AMD\data\repo\onnx.venv\lib\site-packages\diffusers\pipelines\stable_diffusion\convert_from_ckpt.py", line 1301, in convert_controlnet_checkpoint
converted_ctrl_checkpoint = convert_ldm_unet_checkpoint(
File "N:\StableDiffusionUI-Shark-AMD\data\repo\onnx.venv\lib\site-packages\diffusers\pipelines\stable_diffusion\convert_from_ckpt.py", line 380, in convert_ldm_unet_checkpoint
new_checkpoint["time_embedding.linear_1.weight"] = unet_state_dict["time_embed.0.weight"]
KeyError: 'time_embed.0.weight'
This comment has been hidden
Very cool @chris-crucible ,
I've converted the diffusers checkpoint, verified that it works and also added a short example. You can find everything here: https://huggingface.co/CrucibleAI/ControlNetMediaPipeFace/discussions/6
JosephCatrambone
changed discussion status to
closed