ljp commited on
Commit
83f4ced
1 Parent(s): 411df28

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +68 -18
README.md CHANGED
@@ -14,13 +14,13 @@ library_name: diffusers
14
  ---
15
 
16
  <div style="display: flex; justify-content: center; align-items: center;">
17
- <img src="images/alibaba.png" alt="alibaba" style="width: 20%; height: auto; margin-right: 5%;">
18
- <img src="images/alimama.png" alt="alimama" style="width: 20%; height: auto;">
19
  </div>
20
 
21
  # FLUX.1-dev ControlNet Inpainting - Beta
22
 
23
- This repository hosts an improved Inpainting ControlNet checkpoint for the [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) model, developed by the AlimamaCreative Team.
24
 
25
  ## Key Enhancements
26
 
@@ -32,32 +32,77 @@ Our latest inpainting model brings significant improvements compared to the prev
32
 
33
  ## Showcase
34
 
35
- The following images were generated using a ComfyUI workflow with these settings (click here to download):
36
  `control-strength` = 1.0, `control-end-percent` = 1.0, `true_cfg` = 1.0
37
 
38
- | Image & Prompt Input | Alpha Version | Beta Version |
39
- |:---:|:---:|:---:|
40
- | ![Input Image](path/to/original_image.jpg) A > B | ![Alpha](path/to/old_model_result.jpg) | ![Beta](path/to/new_model_result.jpg) |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
 
42
  ### ComfyUI Usage Guidelines:
43
 
44
- Download example ComfyUI workflow [here](https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/alimama-flux-controlnet-inpaint.json).
45
 
46
- - Using `t5xxl-FP16` and `flux1-dev-fp8` models for 28-step inference:
47
  - GPU memory usage: 27GB
48
- - Inference time: 27 seconds (cfg=3.5), 15 seconds (cfg=1)
49
- - For optimal results, experiment with lower values for `control-strength`, `control-end-percent`, and `cfg
 
50
 
51
  | Parameter | Recommended Range | Effect |
52
  |-----------|------------------|--------|
53
- | control-strength | 0.0 - 1.0 | Controls how much influence the ControlNet has on the generation. Higher values result in stronger adherence to the control image. |
54
- | control-end-percent | 0.0 - 1.0 | Determines at which point in the denoising process the ControlNet influence ends. Lower values allow for more creative freedom in later steps. |
55
- | cfg (Classifier-Free Guidance Scale) | 1.0 - 30.0 | Influences how closely the generation follows the prompt. Higher values increase prompt adherence but may reduce image quality. |
56
-
57
- ## Model Specifications
58
 
59
- - Training dataset: 15M images from LAION2B and proprietary sources
60
- - Optimal inference resolution: 1024x1024
61
 
62
  ## Diffusers Integration
63
 
@@ -76,6 +121,11 @@ git clone https://github.com/alimama-creative/FLUX-Controlnet-Inpainting.git
76
  python main.py
77
  ````
78
 
 
 
 
 
 
79
  ## License
80
 
81
  Our model weights are released under the [FLUX.1 [dev]](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md) Non-Commercial License.
 
14
  ---
15
 
16
  <div style="display: flex; justify-content: center; align-items: center;">
17
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/alibaba.png" alt="alibaba" style="width: 20%; height: auto; margin-right: 5%;">
18
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/alimama.png" alt="alimama" style="width: 20%; height: auto;">
19
  </div>
20
 
21
  # FLUX.1-dev ControlNet Inpainting - Beta
22
 
23
+ This repository hosts an improved Inpainting ControlNet checkpoint for the [alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha](https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha) model, developed by the AlimamaCreative Team.
24
 
25
  ## Key Enhancements
26
 
 
32
 
33
  ## Showcase
34
 
35
+ The following images were generated using a ComfyUI workflow ([click here to download](https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/alimama-creative-flux-inapint-cn-beta.json)) with these settings:
36
  `control-strength` = 1.0, `control-end-percent` = 1.0, `true_cfg` = 1.0
37
 
38
+ | Image & Prompt Input | Alpha Version | Enhanced Version |
39
+ |-----------------------------------------------------|----------------------------------------------------|----------------------------------------------------|
40
+
41
+ <div align = "center">
42
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/1_0.png" width = "30%" style="display:inline-block;" />
43
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/1_1.png" width = "30%" style="display:inline-block;" />
44
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/1_2.png" width = "30%" style="display:inline-block;"/>
45
+ <br>
46
+ Prompt : <i>'Write a few lines of words "alimama creative" on the wooden board'</i>
47
+ </div>
48
+
49
+ <div align = "center">
50
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/2_0.png" width = "30%" style="display:inline-block;" />
51
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/2_1.png" width = "30%" style="display:inline-block;" />
52
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/2_2.png" width = "30%" style="display:inline-block;"/>
53
+ <br>
54
+ Prompt : <i>"a girl with big beautiful white wing"</i>
55
+ </div>
56
+
57
+ <div align = "center">
58
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/3_0.png" width = "30%" style="display:inline-block;" />
59
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/3_1.png" width = "30%" style="display:inline-block;" />
60
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/3_2.png" width = "30%" style="display:inline-block;"/>
61
+ <br>
62
+ Prompt : <i>"red hair"</i>
63
+ </div>
64
+
65
+ <div align = "center">
66
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/4_0.png" width = "30%" style="display:inline-block;" />
67
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/4_1.png" width = "30%" style="display:inline-block;" />
68
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/4_2.png" width = "30%" style="display:inline-block;"/>
69
+ <br>
70
+ Prompt : <i>" "</i>
71
+ </div>
72
+
73
+ <div align = "center">
74
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/5_0.png" width = "30%" style="display:inline-block;" />
75
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/5_1.png" width = "30%" style="display:inline-block;" />
76
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/5_2.png" width = "30%" style="display:inline-block;"/>
77
+ <br>
78
+ Prompt : <i>"Albert Einstein"</i>
79
+ </div>
80
+
81
+ <div align = "center">
82
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/6_0.png" width = "30%" style="display:inline-block;" />
83
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/6_1.png" width = "30%" style="display:inline-block;" />
84
+ <img src="https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/6_2.png" width = "30%" style="display:inline-block;"/>
85
+ <br>
86
+ Prompt : <i>"Ravello Outdoor Sectional Sofa Set with Coffee Table"</i>
87
+ </div>
88
 
89
  ### ComfyUI Usage Guidelines:
90
 
91
+ Download example ComfyUI workflow [here](https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Alpha/resolve/main/images/alimama-creative-flux-inapint-cn-beta.json).
92
 
93
+ - Using `t5xxl-FP16` and `flux1-dev-fp8` models for 30-step inference @1024px & H20 GPU:
94
  - GPU memory usage: 27GB
95
+ - Inference time: 48 seconds (true_cfg=3.5), 26 seconds (true_cfg=1)
96
+
97
+ - Different results can be achieved by adjusting the following parameters:
98
 
99
  | Parameter | Recommended Range | Effect |
100
  |-----------|------------------|--------|
101
+ | control-strength | 0.6 - 1.0 | Controls how much influence the ControlNet has on the generation. Higher values result in stronger adherence to the control image. |
102
+ | controlend-percent | 0.35 - 1.0 | Determines at which step in the denoising process the ControlNet influence ends. Lower values allow for more creative freedom in later steps. |
103
+ | true-cfg (Classifier-Free Guidance Scale) | 1.0 or 3.5 | Influences how closely the generation follows the prompt. Higher values increase prompt adherence but may reduce image quality. |
 
 
104
 
105
+ - More comprehensive full-image prompts can lead to better overall results. For example, in addition to describing the area to be repaired, you can also describe the background, atmosphere, and style of the entire image. This approach can make the generated results more harmonious and natural.
 
106
 
107
  ## Diffusers Integration
108
 
 
121
  python main.py
122
  ````
123
 
124
+ ## Model Specifications
125
+
126
+ - Training dataset: 15M images from LAION2B and proprietary sources
127
+ - Optimal inference resolution: 1024x1024
128
+
129
  ## License
130
 
131
  Our model weights are released under the [FLUX.1 [dev]](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md) Non-Commercial License.