Update README.md
Browse files
README.md
CHANGED
@@ -306,17 +306,21 @@ On the other hand, styles trained into LoHas would be less effective if applying
|
|
306 |
![xyz_grid-0030-20230330090242](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/xyz_grid-0030-20230330090242.jpg)
|
307 |
|
308 |
|
309 |
-
|
310 |
|
311 |
To summarize, if you want to have style of some model X, instead of training directly on X it would be better to train on an ancestor of X that does not contain this style.
|
312 |
Therefore, if you want to get cosplay of characters, you can do either of the following
|
313 |
- Train on NED and add a decent amount of photos in regularization set
|
314 |
-
- Train on
|
315 |
|
316 |
Lykon did show some successful results by only training with anime images on NED, but I doubt this is really optimal. Actually, he uses again a doll LoRa to reinforce the photo-realistic concept. It may be simper to just do what I suggest above.
|
317 |
|
|
|
318 |
|
319 |
-
|
|
|
|
|
|
|
320 |
|
321 |
Clearly, the thing that really matters is how the model is made, and not how the model looks like. A model that is versatile in style does not make it a good base model for whatever kind of training. In fact, VBP2-2 has around 300 styles trained in but LoHa trained on top of it does not transfer well to other models.
|
322 |
Similarly, two models that produce similar style do not mean they transfer well to each other. Both MFB and Salt-Mix have strong anime screenshot style but a LoHa trained on MFB does not transfer well to Salt-Mix.
|
@@ -372,6 +376,22 @@ However, you may also need to worry about some bad hand in such a model
|
|
372 |
![00032-20230330225216](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/00032-20230330225216.png)
|
373 |
|
374 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
375 |
|
376 |
#### Training Speed
|
377 |
|
|
|
306 |
![xyz_grid-0030-20230330090242](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/xyz_grid-0030-20230330090242.jpg)
|
307 |
|
308 |
|
309 |
+
*Implication for Making Cosplay Images*
|
310 |
|
311 |
To summarize, if you want to have style of some model X, instead of training directly on X it would be better to train on an ancestor of X that does not contain this style.
|
312 |
Therefore, if you want to get cosplay of characters, you can do either of the following
|
313 |
- Train on NED and add a decent amount of photos in regularization set
|
314 |
+
- Train on NAI and transfer to NED
|
315 |
|
316 |
Lykon did show some successful results by only training with anime images on NED, but I doubt this is really optimal. Actually, he uses again a doll LoRa to reinforce the photo-realistic concept. It may be simper to just do what I suggest above.
|
317 |
|
318 |
+
For illustration, here is what we get by applying the LoHas trained on different base models on NeverEndingDream. As we can see, the one trained on NAI retains the most photo-realistic style (of course you can always adjusting various weight to improve things but I still get the best result with the one trained on NAI).
|
319 |
|
320 |
+
![xyz_grid-0017-20230331061354](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/xyz_grid-0017-20230331061354.jpg)
|
321 |
+
|
322 |
+
|
323 |
+
*Some Myths*
|
324 |
|
325 |
Clearly, the thing that really matters is how the model is made, and not how the model looks like. A model that is versatile in style does not make it a good base model for whatever kind of training. In fact, VBP2-2 has around 300 styles trained in but LoHa trained on top of it does not transfer well to other models.
|
326 |
Similarly, two models that produce similar style do not mean they transfer well to each other. Both MFB and Salt-Mix have strong anime screenshot style but a LoHa trained on MFB does not transfer well to Salt-Mix.
|
|
|
376 |
![00032-20230330225216](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/00032-20230330225216.png)
|
377 |
|
378 |
|
379 |
+
**A Case Study on Training on vanilla SD Model**
|
380 |
+
|
381 |
+
Someone may believe that training on vanilla SD model would give better result. I however not advice it for anime training because you can hardly make your model usable with a small dataset. In turn you get a model that does not understand booru tags and often produces bad anatomy and bad hands.
|
382 |
+
|
383 |
+
Moreover, as we see below, all the vanilla SD models are so far away from NAI that even character cannot be transfer. It would only transfer to more photo-realistic model such as ChilloutMix (as they have a more important vanilla SD component) but this kills their photo-realistic style and make something terrible so this really there is really no interests in so doing in general.
|
384 |
+
|
385 |
+
The trend we see here is also coherent as what we have observed so far
|
386 |
+
- When training on vanilla SD and applying to descendant model we get mostly the style of descendant model
|
387 |
+
- The only exception is ChilloutMix for which it becomes less photo-realistic, which makes sense because it is much closer to vanilla SD
|
388 |
+
- Training on NAI and applying to vanilla SD gives a diluted anime style or no style at all
|
389 |
+
|
390 |
+
![xyz_grid-0016-20230331054835](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/xyz_grid-0016-20230331054835.jpg)
|
391 |
+
![xyz_grid-0014-20230331053913](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/xyz_grid-0014-20230331053913.jpg)
|
392 |
+
![xyz_grid-0015-20230331054327](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/xyz_grid-0015-20230331054327.jpg)
|
393 |
+
|
394 |
+
|
395 |
|
396 |
#### Training Speed
|
397 |
|