|
--- |
|
license: creativeml-openrail-m |
|
tags: |
|
- stable-diffusion |
|
- anime |
|
- aiart |
|
pipeline_tag: text-to-image |
|
--- |
|
|
|
**This model intends to be the ultimate uma-musume model where we try to train as many things about uma-musume as possible into it.** |
|
Ok it's not true. Only characters and outfits are trained, ant not locations/objects/running-style(?) or whatever. |
|
|
|
|
|
## Example Generations |
|
|
|
See https://civitai.com/models/13543/umamusume |
|
|
|
|
|
## Usage |
|
|
|
You don't need to use the model as is. Treat it as lora and do either |
|
|
|
- **Your favorite model + w * (umamusume model - ACertainty/Nai)** |
|
- or **Umamusume model + w * (your favorite model - ACertainty/Nai)** |
|
|
|
|
|
#### How to prompt |
|
|
|
Here are two example captions that are used for training |
|
|
|
> AgnesDigital; CurrenChan, tracen school uniform; fanart; 2girls, horse girl, multiple girls, character doll, phone, beads, bangs, bow, blush, animal ears, school uniform, holding phone, two side up, heart in mouth, one eye closed, white background, sweat, tracen school uniform, selfie, red bow, shirt, sailor collar, open mouth, simple background, heart, horse ears, holding, purple shirt, purple skirt, skirt, long sleeves |
|
|
|
> CurrenChan; SmartFalcon, racing suit; fanart; 2girls, horse girl, multiple girls, dress, diamond (shape), party, nail polish, red bow, see-through, teeth, frilled dress, upper teeth only, balloon, blurry, frilled collar, animal ears, timestamp, ring, black bow, pink dress, border, gem, short sleeves, puffy sleeves, heart hands, pink nails, strap, bow, lace-trimmed sleeves, black headband, chromatic aberration, white border, black dress, writing on wall, headband, frills, horse tail, chain, horse ears, puffy short sleeves, bracelet, tail, bangs, open mouth, collar, black nails, multicolored dress, multicolored clothes, depth of field, heart |
|
|
|
## Concepts |
|
|
|
#### List of characters |
|
|
|
**Umamusume** |
|
|
|
- AdmireVega (アドマイヤベガ 愛慕織姬) |
|
- AgnesDigital (アグネスデジタル 愛麗數碼) |
|
- AgnesTachyon (アグネスタキオン 愛麗速子) |
|
- AirGroove (エアグルーヴ 氣槽) |
|
- AirShakur (エアシャカール 空中神宮) |
|
- AstonMachan (アストンマーチャン 真弓快車) |
|
- BambooMemory (バンブーメモリー 青竹回憶) |
|
- BikoPegasus (ビコーペガサス 微光飛駒) |
|
- BiwaHayahide (ビワハヤヒデ 琵琶晨光) |
|
- Broye (ブロワイエ 望族) |
|
- ChevalGrand (シュヴァルグラン 高尚駿逸) |
|
- CopanoRickey (コパノリッキー 小林歷奇) |
|
- CurrenChan (カレンチャン 真機伶) |
|
- DaiichiRuby (ダイイチルビー 第一紅寶) |
|
- DaitakuHelios (ダイタクヘリオス 大拓太陽神) |
|
- DaiwaScarlet (ダイワスカーレット 大和赤驥) |
|
- DaringTact (デアリングタクト 謀勇兼備) |
|
- EishinFlash (エイシンフラッシュ 榮進閃耀) |
|
- ElCondorPasa (エルコンドルパサー 神鷹) |
|
- FineMotion (ファインモーション 美妙姿勢) |
|
- FujiKiseki (フジキセキ 富士奇石) |
|
- GoldCity (ゴールドシチー 黃金城市) |
|
- GoldShip (ゴールドシップ 黃金船) |
|
- GrassWonder (グラスワンダー 草上飛) |
|
- HappyMeek (HappyMeek 快樂米可) |
|
- HaruUrara (ハルウララ 春烏菈菈) |
|
- HishiAkebono (ヒシアケボノ 菱曙) |
|
- HishiAmazon (ヒシアマゾン 菱亞馬遜) |
|
- HokkoTarumae (ホッコータルマエ 北港火山) |
|
- IkunoDictus (イクノディクタス 狄杜斯) |
|
- InariOne (イナリワン 稻荷一) |
|
- InesFujin (アイネスフウジン 艾尼斯風神) |
|
- KSMiracle (ケイエスミラクル 神奇駒) |
|
- KawakamiPrincess (カワカミプリンセス 川上公主) |
|
- KingHalo (キングヘイロー 帝王光輝) |
|
- KitasanBlack (キタサンブラック 北部玄駒) |
|
- LittleCocon (リトルココン 小巧圓繭) |
|
- ManhattanCafe (マンハッタンカフェ 曼城茶座) |
|
- Maruzensky (マルゼンスキー 丸善斯基) |
|
- MarvelousSunday (マーベラスサンデー 美麗週日) |
|
- MatikaneTannhauser (マチカネタンホイザ 詩歌劇) |
|
- Matikanefukukitaru (マチカネフクキタル 待兼福來) |
|
- MayanoTopGun (マヤノトップガン 重砲) |
|
- MeishoDoto (メイショウドトウ 名將怒濤) |
|
- MejiroArdan (メジロアルダン 目白阿爾丹) |
|
- MejiroBright (メジロブライト 目白光明) |
|
- MejiroDober (メジロドーベル 目白多伯) |
|
- MejiroMcQueen (メジロマックイーン 目白麥昆) |
|
- MejiroPalmer (メジロパーマー 目白善信) |
|
- MejiroRamonu (メジロラモーヌ 目白山峰) |
|
- MejiroRyan (メジロライアン 目白賴恩) |
|
- MihonoBourbon (ミホノブルボン 美蒲波旁) |
|
- MrCB (ミスターシービー CB先生) |
|
- NakayamaFesta (ナカヤマフェスタ 中山慶典) |
|
- NaritaBrian (ナリタブライアン 成田白仁) |
|
- NaritaTaishin (ナリタタイシン 成田大進) |
|
- NaritaTopRoad (ナリタトップロード 成田路) |
|
- NiceNature (ナイスネイチャ 優秀素質) |
|
- NishinoFlower (ニシノフラワー 西野花) |
|
- OguriCap (オグリキャップ 小栗帽) |
|
- Orfevre (オルフェーヴル 黃金巨匠) |
|
- RiceShower (ライスシャワー 米浴) |
|
- SakuraBakushinO (サクラバクシンオー 櫻花進王) |
|
- SakuraChiyonoO (サクラチヨノオー 櫻花千代王) |
|
- SakuraLaurel (サクラローレル 櫻花桂冠) |
|
- SatonoDiamond (サトノダイヤモンド 里見光鑽 |
|
- SeiunSky (セイウンスカイ 星雲天空) |
|
- ShinkoWindy (シンコウウインディ 新光風) |
|
- SilenceSuzuka (サイレンススズカ 無聲鈴鹿) |
|
- SiriusSymboli (シリウスシンボリ 天狼象徵) |
|
- SmartFalcon (スマートファルコン 醒目飛鷹) |
|
- SunVisor (サンバイザー 太陽耀斑) |
|
- SpecialWeek (ペシャルウィーク 特別週) |
|
- SuperCreek (スーパークリーク 超級小海灣) |
|
- SweepTosho (スイープトウショウ 東商變革) |
|
- SymboliKrisS (シンボリクリスエス 吉兆) |
|
- Symboli Rudolf (シンボリルドルフ 皇帝) |
|
- TMOperaO (テイエムオペラオー 好歌劇) |
|
- TaikiShuttle (タイキシャトル 大樹快車) |
|
- TamamoCross (タマモクロス 玉藻十字) |
|
- TaninoGimlet (タニノギムレット 谷野美酒) |
|
- TokaiTeio (トウカイテイオー 東海帝皇) |
|
- TosenJordan (トーセンジョーダン 東瀛佐敦) |
|
- TsurumaruTsuyoshi (ツルマルツヨシ 鶴丸剛志) |
|
- TwinTurbo (ツインターボ 雙渦輪) |
|
- Vodka (ウオッカ 伏特加) |
|
- WinningTicket (ウイニングチケット 勝利獎券) |
|
- WonderAcute (ワンダーアキュート 奇銳駿) |
|
- YaenoMuteki (ヤエノムテキ 八重無敵) |
|
- YamaninZephyr (ヤマニンゼファー 也文攝輝) |
|
- YukinoBijin (ユキノビジン 雪之美人) |
|
- ZennoRobRoy (ゼンノロブロイ 荒漠英雄) |
|
|
|
**Others** |
|
|
|
- AkikawaYayoi (秋川やよい 秋川彌生) \[President of Tracen Academy トレセン学園の理事長\] |
|
- ToujouHana (東条ハナ 東條華) |
|
- SpicaTrainer (沖野T/チームスピカのトレーナー Spica訓練員) |
|
- HayakawaTazuna (駿川たづな 駿川手綱) -- it seems that I kind of mess up and use the tag TazunaHayakawa when she appears in anime |
|
|
|
|
|
#### List of outfits |
|
|
|
**Common outfits** |
|
|
|
- Tracen school uniform |
|
- Tracen gym uniform |
|
- Tracen race uniform |
|
- Tracen swimsuit |
|
- Stage clothes |
|
|
|
**Character specific outfit** |
|
|
|
Racing suit (勝負服) for most characters, casual outfit for some, and also other specific costumes if they are tagged by booru. |
|
You should prompt directly with something like `character, racing suit` and you may need other trigger words for better results. |
|
|
|
|
|
#### Styles |
|
|
|
- aniscreen |
|
- fanart |
|
- support card |
|
|
|
For richer style consider using style prompt that the model usually knows or merging with other models |
|
|
|
|
|
## Dataset description |
|
|
|
Around [60K images](https://huggingface.co/datasets/alea31415/umamusume_private/tree/main/training) containing |
|
- 14327 anime screenshots |
|
- 21377 fan arts |
|
- 270 support cards |
|
- 7822 resused images: From the above three 4019 images are selected to form a few-shot classifier training set, areas centered at face are cropped out which give 3803 images |
|
- 19279 regularization images |
|
|
|
|
|
## Training |
|
|
|
The model is trained in two phases (resolution 512, clip skip 1) on top of [ACertainty](https://huggingface.co/JosephusCheung/ACertainty) |
|
|
|
**The first phase is trained with [EveryDream2](https://github.com/victorchall/EveryDream2trainer)** |
|
|
|
- Specific weighting scheme with maximum repeat 50 |
|
- consine scheduler, 2.5e-6 learning rate |
|
- conditional dropout 0.08 |
|
|
|
Total steps x batch is around 1.1M |
|
- batch 8 for 61495 steps |
|
- batch 6 for ~3500 steps but this gets stopped due to some technical issues |
|
- batch 4 for 150621 steps |
|
|
|
**The second phase is trained with [naifu trainer](https://github.com/AdjointOperator/naifu-diffusion)** |
|
|
|
The goal is to train embeddings for character specific outfits. |
|
Both unets and embeddings are trained but not text encoders. |
|
This phase does not use the anime screenshots nor the regularization images. |
|
In the end, it turns out that although the model is not further trained for "charactar, racing suit" this acutally gets improved and gives more satisfying results than the embeddings. |
|
This is really mysterious and worths more investigation. |
