|
--- |
|
language: '-en' |
|
license: other |
|
tags: |
|
- stable-diffusion |
|
- text-to-image |
|
- safetensors |
|
datasets: nakayama/DeDeDeDataset |
|
--- |
|
# ご利用の際は下記のライセンス内容を十分にご確認ください。 |
|
HonyararaMixはアニメ調の人物を出力しやすいように調整されたStable Diffusion 2.1 解像度768対応版モデルです。 |
|
ベースモデルの[Illuminati Diffusion 1.0](https://huggingface.co/IlluminatiAI/Illuminati_Diffusion_v1.0)へ[Waifu Diffusion 1.5 Beta](https://huggingface.co/waifu-diffusion/wd-1-5-beta) 、[Waifu Diffusion 1.5 Beta2](https://huggingface.co/waifu-diffusion/wd-1-5-beta2)、[Replicant-V1.0](https://huggingface.co/gsdf/Replicant-V1.0)、[Stable Diffusion 2.1](https://huggingface.co/stabilityai/stable-diffusion-2-1)を用いマージしたものをベースに |
|
[DeDeDeDataset](https://huggingface.co/datasets/nakayama/DeDeDeDataset)による学習などを行い調整したものです。 |
|
|
|
旧HonyararaMixにはライセンス上の問題があったため、それらを解決したモデルになります。 |
|
|
|
Prompt/Bad Promptについては、ベースモデルとなった二種のモデルが用意した文書の内容、およびTI Embeddingがこのモデルでも作用します。 |
|
WDについては[こちら](https://cafeai.notion.site/WD-1-5-232cfa9443fc4661b469e7d1df8b7dd8)を参照。Illuminati DiffusionについてはNfixerが割とよく効く印象です。 |
|
|
|
# 例 |
|
<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image01.png" style="max-width:1000px;" width="75%"/> |
|
|
|
``` |
|
best quality, masterpiece, great nature and secret garden with anime 1girl with twintail green hair and green eye and beautiful iris and (((from behind))) |
|
Negative prompt: nfixer,3d, flat shading, flat shadow, bad anatomy, bad hands, missing fingers, extra digit, fewer digits looking at viewer |
|
Steps: 28, Sampler: DDIM, CFG scale: 9, Seed: 43662843, Size: 1024x768, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B |
|
``` |
|
|
|
<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image02.png" style="max-width:1000px;" width="75%"/> |
|
|
|
``` |
|
newest, masterpiece, exceptional, anime game cg of cute cute cute 1girl sitting on a bench in harajuku park with detailed wavy pink hair pink and detailed yellow hair |
|
Negative prompt: nfixer, 3d, flat shading, flat shadow, inaccurate limb, bad hands, missing fingers, extra digit, fewer digits |
|
Steps: 28, Sampler: DPM++ SDE Karras, CFG scale: 7.5, Seed: 4179369813, Size: 768x1152, Model hash: 4dbf24eef6, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B |
|
``` |
|
|
|
<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image03.png" style="max-width:1000px;" width="75%"/> |
|
|
|
``` |
|
newest, masterpiece, exceptional, anime game cg of kawaii samurai 1girl, fighting form and (((intricate hands))), fist and dynamic angle with detailed pink hair and detailed yellow eye and beautiful iris in japanese background |
|
Negative prompt: nfixer, 3d, flat shading, flat shadow, ((bad hands, missing fingers, extra digit, fewer digits)) |
|
Steps: 16, Sampler: DPM++ SDE Karras, CFG scale: 7.5, Seed: 2246223688, Size: 768x1152, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B |
|
``` |
|
|
|
<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image04.png" style="max-width:1000px;" width="75%"/> |
|
|
|
``` |
|
((masterpiece, best quality)),best aesthetic, 1girl, solo, black skirt, blue eyes, guitar, headphones, instrument, long hair, music, one side up, pink hair, playing guiter, pleated skirt, black shirt, indoors |
|
Negative prompt: nfixer,missing finger, extra digits, fewer digits,((mutated hands and fingers)), 3d, flat shadow |
|
Steps: 16, Sampler: DPM++ SDE Karras, CFG scale: 7, Seed: 3689716537, Size: 768x1152, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B |
|
``` |
|
|
|
<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image05.png" style="max-width:1000px;" width="75%"/> |
|
|
|
``` |
|
newest, masterpiece, exceptional, anime game cg of teenage 1boy wizard bust shot casting fire magic spell in New York City, detailed eye and beautiful iris,picturesque, golden hour, dynamic pose |
|
Negative prompt: nfixer, 3d, flat shading, flat shadow, inaccurate limb, bad hands, missing fingers, extra digit, fewer digits |
|
Steps: 25, Sampler: Euler a, CFG scale: 7.5, Seed: 1414427875, Size: 768x1152, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B |
|
``` |
|
|
|
<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image06.png" style="max-width:1000px;" width="75%"/> |
|
|
|
``` |
|
newest, masterpiece, exceptional, anime game cg of samurai (((((old man))))), fighting pose, dynamic angle, fist with detailed pink hair and detailed yellow eye and beautiful iris in japanese background |
|
nfixer, 3d, flat shading, flat shadow, inaccurate limb, bad hands, missing fingers, extra digit, fewer digits |
|
Steps: 25, Sampler: DDIM, CFG scale: 9, Seed: 3970412380, Size: 768x1152, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B |
|
``` |
|
|
|
# マージ・学習手順について |
|
1. Illuminati Diffusion 1.0にWaifu Diffusion 1.5 Beta1を差分マージする |
|
| Interpolation Method | Primary Model | Secondary Model | Tertiary Model | Merge Name | |
|
| --- | --- | --- | --- | --- | |
|
| Add Difference @ 0.6 | Illuminati Diffusion 1.0 | Waifu Diffusion 1.5 Beta1 | Stable Diffusion v2-1_768-ema-pruned | HonyararaMix_pre1 | |
|
|
|
2. HonyararaMix_pre1に対し、[自前で用意した他Diffusion Modelの出力からなる素材画像](https://huggingface.co/datasets/nakayama/DeDeDeDataset)にて学習させる |
|
用意の際に利用したサービス/モデルは、SD2.1、Novel AI、WD1.3/1.4、CoolJapan Diffusion 2.1、Dreamlike Photoreal 2.0。 |
|
使用したソフトは[kohya版Fine Tuning](https://github.com/kohya-ss/sd-scripts)。 |
|
総数は30000程、flipしたものと合わせてlearning rateは5e-6、以下の設定で学習させ、 |
|
試行錯誤の後にepoch 2相当のものをHonyararaMix_pre2とした。 |
|
|
|
``` |
|
accelerate launch --num_cpu_threads_per_process 1 fine_tune.py |
|
--pretrained_model_name_or_path=Honyarara_pre1.safetensors |
|
--in_json meta_lat.json --train_data_dir=train_data |
|
--output_dir=fine_tuned |
|
--shuffle_caption |
|
--train_batch_size=4 |
|
--learning_rate=5e-6 |
|
--max_train_steps=60000 |
|
--v2 |
|
--v_parameterization |
|
--use_8bit_adam |
|
--xformers |
|
--mixed_precision=bf16 |
|
--save_every_n_epochs=2 |
|
--save_precision=float |
|
--max_token_length=150 |
|
--seed=42 |
|
``` |
|
|
|
3. Honyarara_pre1にHonyarara_pre2を加重平均でマージする |
|
| Interpolation Method | Primary Model | Secondary Model | Merge Name | |
|
| --- | --- | --- | --- | |
|
| Weighted Sum @ 0.5 | HonyararaMix_pre1 | HonyararaMix_pre2 | HonyararaMix_pre3 | |
|
|
|
4. HonyararaMix_pre3にIlluminati Diffusion 1.0を階層マージする |
|
使用マージ用スクリプトは[Super Merger](https://github.com/hako-mikan/sd-webui-supermerger)。 |
|
| Model: A | Model: B | Weight | Merge Name | |
|
| --- | --- | --- | --- | |
|
| HonyararaMix_pre3 | Illuminati Diffusion 1.0 | 0.5,0.45,0.0,0.0,0.0,0.0,0.0,0.1,0.3,0.5,0.7,0.9,1.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1,0.3,0.5,0.7,0.9,1.0 | HonyararaMix_pre4| |
|
|
|
5. HonyararaMix_pre4にWaifu Diffusion 1.5 Beta2 Aestheticを差分マージする |
|
| Interpolation Method | Primary Model | Secondary Model | Tertiary Model | Merge Name | |
|
| --- | --- | --- | --- | --- | |
|
| Add Difference @ 0.5 | HonyararaMix_pre4 | WD 1.5 Beta 2 Aesthetic | WD 1.5 Beta 2 | HonyararaMix_pre5 | |
|
|
|
6. HonyararaMix_pre5にReplicant-V1.0を加重平均でマージする |
|
| Interpolation Method | Primary Model | Secondary Model | Merge Name | |
|
| --- | --- | --- | --- | |
|
| Weighted Sum @ 0.5 | HonyararaMix_pre5 | Replicant-V1.0 | HonyararaMix A | |
|
|
|
7. HonyararaMix AにHonyararaMix_pre4を加重平均でマージする |
|
| Interpolation Method | Primary Model | Secondary Model | Merge Name | |
|
| --- | --- | --- | --- | |
|
| Weighted Sum @ 0.5 | HonyararaMix A | HonyararaMix_pre4 | HonyararaMix B | |
|
|
|
# ライセンスについて |
|
|
|
Fair AI Public License 1.0-SD (https://freedevproject.org/faipl-1.0-sd/) |
