File size: 2,783 Bytes
76b45c8 b7cca8c 8e2a773 b7cca8c e4f4bc8 76b45c8 b7cca8c 180e2dc b7cca8c b4aec14 b7cca8c 8e2a773 b7cca8c 76b45c8 b7cca8c 8e2a773 b7cca8c 76b45c8 8e2a773 b7cca8c 8e2a773 76b45c8 b7cca8c 3da89c9 8698e97 b7cca8c 76b45c8 b4aec14 b7cca8c 76b45c8 b7cca8c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 |
---
library_name: diffusers
base_model: stabilityai/stable-diffusion-xl-base-1.0
tags:
- lora
- text-to-image
license: openrail++
inference: false
---
# Latent Consistency Model (LCM) LoRA: SDXL
Latent Consistency Model (LCM) LoRA was proposed in [LCM-LoRA: A universal Stable-Diffusion Acceleration Module](https://arxiv.org/abs/2311.05556)
by *Simian Luo, Yiqin Tan, Suraj Patil, Daniel Gu et al.*
It is a distilled consistency adapter for [`stable-diffusion-xl-base-1.0`](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) that allows
to reduce the number of inference steps to only between **2 - 8 steps**.
| Model | Params / M |
|----------------------------------------------------------------------------|------------|
| [lcm-lora-sdv1-5](https://huggingface.co/latent-consistency/lcm-lora-sdv1-5) | 67.5 |
| [lcm-lora-ssd-1b](https://huggingface.co/latent-consistency/lcm-lora-ssd-1b) | 105 |
| [**lcm-lora-sdxl**](https://huggingface.co/latent-consistency/lcm-lora-sdxl) | **197M** |
## Usage
LCM-LoRA is supported in 🤗 Hugging Face Diffusers library from version v0.23.0 onwards. To run the model, first
install the latest version of the Diffusers library as well as `peft`, `accelerate` and `transformers`.
audio dataset from the Hugging Face Hub:
```bash
pip install --upgrade pip
pip install --upgrade diffusers transformers accelerate peft
```
### Text-to-Image
The adapter can be loaded with it's base model `stabilityai/stable-diffusion-xl-base-1.0`. Next, the scheduler needs to be changed to [`LCMScheduler`](https://huggingface.co/docs/diffusers/v0.22.3/en/api/schedulers/lcm#diffusers.LCMScheduler) and we can reduce the number of inference steps to just 2 to 8 steps.
Please make sure to either disable `guidance_scale` or use values between 1.0 and 2.0.
```python
import torch
from diffusers import LCMScheduler, AutoPipelineForText2Image
model_id = "stabilityai/stable-diffusion-xl-base-1.0"
adapter_id = "latent-consistency/lcm-lora-sdxl"
pipe = AutoPipelineForText2Image.from_pretrained(model_id, torch_dtype=torch.float16, variant="fp16")
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)
pipe.to("cuda")
# load and fuse lcm lora
pipe.load_lora_weights(adapter_id)
pipe.fuse_lora()
prompt = "Self-portrait oil painting, a beautiful cyborg with golden hair, 8k"
# disable guidance_scale by passing 0
image = pipe(prompt=prompt, num_inference_steps=4, guidance_scale=0).images[0]
```
### Image-to-Image
Works as well! TODO docs
### Inpainting
Works as well! TODO docs
### ControlNet
Works as well! TODO docs
### T2I Adapter
Works as well! TODO docs
## Speed Benchmark
TODO
## Training
TODO
|