chaofengc commited on
Commit
949c735
1 Parent(s): 374330d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +29 -0
README.md CHANGED
@@ -1,3 +1,32 @@
1
  ---
2
  license: cc-by-nc-sa-4.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-nc-sa-4.0
3
  ---
4
+
5
+ # Enhancing Diffusion Models with Text-Encoder Reinforcement Learning
6
+
7
+ Official PyTorch codes for paper [Enhancing Diffusion Models with Text-Encoder Reinforcement Learning](https://arxiv.org/abs/2311.15657)
8
+
9
+ ## Results on SDXL-Turbo
10
+
11
+ We also applied our method to the recent model [sdxl-turbo](https://huggingface.co/stabilityai/sdxl-turbo). The model is trained with [Q-Instruct](https://github.com/Q-Future/Q-Instruct) feedback through direct back-propagation to save training time. Test with the following codes
12
+
13
+ ```
14
+ ## Note: sdturbo requires latest diffusers installed from source with the following command
15
+ git clone https://github.com/huggingface/diffusers
16
+ cd diffusers
17
+ pip install -e .
18
+ ```
19
+ ```
20
+ from diffusers import AutoPipelineForText2Image
21
+ import torch
22
+
23
+ pipe = AutoPipelineForText2Image.from_pretrained("stabilityai/sd-turbo", torch_dtype=torch.float16, variant="fp16")
24
+ pipe = pipe.to("cuda")
25
+ pipe.load_lora_weights('chaofengc/sdxl-turbo_texforce')
26
+
27
+ pt = ['a photo of a cat.']
28
+ img = pipe(prompt=pt, num_inference_steps=1, guidance_scale=0.0).images[0]
29
+ ```
30
+
31
+ ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6304798d41387c7f117558f7/MjIO6YL6RufTwmPTNIUBB.jpeg)
32
+