stabilityai

/

stable-diffusion-2

like 1.83k

Follow

Stability AI 5,891

I too am having this issue.

Where did you even figure out how to get that far?

@clayshoaf This hero right here. https://github.com/JoePenna/Dreambooth-Stable-Diffusion#using-the-generated-model

This could help https://www.reddit.com/r/StableDiffusion/comments/zlaec2/help_img2img_inpainting_error_with_sd_v21/

By doing something similar to the reddit post above I am able to run it without errors but something is not set up right the images are randomly altered of the original as to be unrecognizable.

pipestart = StableDiffusionImg2ImgPipeline.from_pretrained(
    "stabilityai/stable-diffusion-2",
    revision="fp16",
    torch_dtype=torch.float16,
)
pipe = StableDiffusionImg2ImgPipeline(**pipestart.components).to("cuda")

@juggernaught I think you need to import StableDiffusionPipeline first, where you use pretrained
https://huggingface.co/docs/diffusers/main/en/api/pipelines/stable_diffusion/overview#how-to-convert-all-use-cases-with-multiple-or-single-pipeline

It is further up in the code. I think this is all of it. Uplaod test1.jpg into root.

!pip install -Uq diffusers transformers fastcore

import logging
from pathlib import Path

import matplotlib.pyplot as plt
import torch
from diffusers import StableDiffusionPipeline
from fastcore.all import concat
from huggingface_hub import notebook_login
from PIL import Image

logging.disable(logging.WARNING)

torch.manual_seed(1)
if not (Path.home()/'.huggingface'/'token').exists(): notebook_login()

from diffusers import StableDiffusionImg2ImgPipeline
from fastdownload import FastDownload

def image_grid(imgs, rows, cols):
    w,h = imgs[0].size
    grid = Image.new('RGB', size=(cols*w, rows*h))
    for i, img in enumerate(imgs): grid.paste(img, box=(i%cols*w, i//cols*h))
    return grid

pipestart = StableDiffusionImg2ImgPipeline.from_pretrained(
    "stabilityai/stable-diffusion-2",
    revision="fp16",
    torch_dtype=torch.float16,
).to("cuda") #making this have .to("cuda") also made no difference.
pipe = StableDiffusionImg2ImgPipeline(**pipestart.components).to("cuda")

init_image = Image.open("test1.jpg").convert("RGB")

torch.manual_seed(1000)
prompt = "a lovely sunset"
images = pipe(prompt=prompt, num_images_per_prompt=3, init_image=init_image).images
image_grid(images, rows=1, cols=3)

Ok. I mean, not use StableDiffusionImg2ImgPipeline.from_pretrained but StableDiffusionPipeline.from_pretrained

pipestart = StableDiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-2",
    revision="fp16",
    torch_dtype=torch.float16,
).to("cuda") #making this have .to("cuda") also made no difference.
pipe = StableDiffusionImg2ImgPipeline(**pipestart.components).to("cuda")

or try to use stable diffusion mega: https://github.com/huggingface/diffusers/tree/main/examples/community#stable-diffusion-mega

@curibe it made no difference.