baizh0u commited on
Commit
241d9f8
1 Parent(s): 887a0db

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +102 -5
README.md CHANGED
@@ -1,5 +1,102 @@
1
- ---
2
- license: other
3
- license_name: flux-1-dev-non-commercial-license
4
- license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md
5
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: flux-1-dev-non-commercial-license
4
+ license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md
5
+ base_model: black-forest-labs/FLUX.1-dev
6
+ pipeline_tag: text-to-image
7
+ ---
8
+ ![Intro Image](assets/open_source.png)
9
+ ## Summary
10
+ by TheMisto.ai @Shenzhen, China
11
+ This is a ControlNet network designed for any lineart or outline sketches, compatible with Flux1.dev. The ControlNet model parameters are approximately 1.4B.
12
+
13
+ This model is not compatible with XLabs loaders and samplers. Please use TheMisto.ai Flux ControlNet ComfyUI suite.
14
+ This is a Flow matching structure Flux-dev model, utilizing a scalable Transformer module as the backbone of this ControlNet.
15
+
16
+ We've implemented a dual-stream Transformer structure, which enhances alignment and expressiveness for various types of lineart and outline conditions without increasing inference time. The model has also been trained for alignment with both T5 and clip-l TextEncoders, ensuring balanced performance between conditioning images and text prompts.
17
+ For more details on the Flux.dev model structure, visit: https://huggingface.co/black-forest-labs/FLUX.1-dev
18
+
19
+ This ControlNet is compatible with Flux1.dev's fp16/fp8 and other models quantized with Flux1.dev. ByteDance 8/16-step distilled models have not been tested.
20
+ - The example workflow uses the flux1-dev-Q4_K_S.gguf quantized model.
21
+ - Generation quality: Flux1.dev(fp16)>>Flux1.dev(fp8)>>Other quantized models
22
+ - Generation speed: Flux1.dev(fp16)<<< Flux1.dev(fp8) <<< Other quantized models
23
+
24
+ ## Performance
25
+ ### Performance Across Different Sizes and Scenarios
26
+ Tested in various common scenarios such as industrial design, architecture, interior design, animation, games, and photography.
27
+ Make sure to craft your prompts well—precision is more important than length!
28
+ Performance examples are shown below:
29
+ ![Result2](assets/result2.jpg)
30
+
31
+ ### Performance with Different Lineart or Scribble Preprocessors
32
+ Test Parameters:
33
+ - Prompt: "Hyper-realistic 3D render of a classic Formula 1 race car, bright red with Marlboro and Agip logos, number 1, large black tires, dramatic studio lighting, dark moody background, reflective floor, cinematic atmosphere, Octane render style, high detail"
34
+ - controlnet_strength: 0.650.8 (Recommended: Anyline with 0.60.7)
35
+ - steps: 30
36
+ - guidance: 4.0
37
+ - The quality of generated images is positively correlated with prompt quality. Controlnet_strength may vary for different types of lineart and outlines, so experiment with the settings!
38
+ ![Result1](assets/result1.jpg)
39
+
40
+ ### Recommended Settings
41
+ - Image resolution: 720px or above on the short edge
42
+ - controlnet strength: 0.6~0.85 (adjust as needed)
43
+ - guidance: 3.0~5.0 (adjust as needed)
44
+ - steps: 30 or more
45
+
46
+ ## Huggingface (抱抱脸):
47
+ [MistoLine_Flux.dev_v1](https://huggingface.co/TheMistoAI/MistoLine_Flux.dev)
48
+
49
+ ## Usage
50
+ - Download the model from [MistoLine_Flux.dev_v1](https://huggingface.co/TheMistoAI/MistoLine_Flux.dev)
51
+ - Place the model in the ComfyUI\models\TheMisto_model\ directory
52
+ - The directory will be automatically created the first time you run the ComfyUI's TheMisto.ai Flux ControlNet ComfyUI suite
53
+ - Run using ComfyUI; an example workflow can be found in the workflow folder
54
+ - Note: The length and width of the conditioning image must be divisible by 16, or an error will occur.
55
+ ### ComfyUI
56
+ ![ComfyUI-workflow](assets/comfyui.png)
57
+
58
+ ## Training Details
59
+ The Transformer structure, under the scale law, has a significant impact on training time and computational power (higher compute cost, longer training time).
60
+ The training cost for MistoLine_Flux1_dev is several times that of MistoLineSDXL.
61
+ We conducted extensive ablation experiments to balance performance with training costs.
62
+ This training was done using A100-80GB with bf16 mixed precision on the Flux1.dev series models. Apart from Lora, consumer-grade GPUs are basically unsuitable for training.
63
+ In our experiments with larger parameter models, multi-GPUs, multi-node parallel training was required, which is costly.
64
+ If we reach 50,000 stars, we will open-source the Technical Report detailing more training details.
65
+
66
+ ## License
67
+ Align to the FLUX.1 [dev] Non-Commercial License
68
+ This ComfyUI node falls under ComfyUI
69
+ This model is for research and educational purposes only and may not be used for any form of commercial purposes.
70
+
71
+ ## Business Cooperation
72
+ For any custom model training, commercial cooperation, AI application development, or other business collaboration matters, please contact us.
73
+
74
+ - Business: info@themisto.ai
75
+ - Investment: investment@themisto.ai
76
+ We also welcome investors to inquire for more information.
77
+
78
+ Business Email: info@themisto.ai
79
+ Investor Relations: investment@themisto.ai
80
+ ## WIP
81
+ - Flux1.dev-MistoCN-collection
82
+ - Flux1.dev-Misto-IPAdapter
83
+ Your support is the driving force behind our open-source efforts!
84
+
85
+ ## One more thing
86
+ ![Product](assets/misto.png)
87
+ We will soon be launching our own product: An extremely user-friendly multi-modal AI creative tool - [Misto]
88
+ Designed to rekindle the public's creative desire through the simplest and most inspiring experience.
89
+ Unleash creativity, expand the boundaries of imagination, and turn endless inspiration into reality!
90
+
91
+ Supported Platforms: All Platforms
92
+
93
+ ## Media
94
+ ### International:
95
+ Website: https://themisto.ai/
96
+ Discord: https://discord.gg/fTyDB2CU
97
+ X: https://x.com/AiThemisto79359
98
+
99
+ ### Mainland China (中国大陆):
100
+ Website: https://themisto.ai/
101
+ WeChat Official Account: TheMisto AI (Shenzhen Mixed Tuple Technology Co., Ltd.)
102
+ Xiaohongshu: TheMisto.ai (Xiaohongshu ID: 4211745997)