LoCon Testing
https://github.com/KohakuBlueleaf/LoCon
Dim/Alpha
Testing with conv: 1dim 1alpha (due to coding error the examples are using 4alpha, has since been fixed in the code base) linear(network): 4dim 4alpha We will refer to dim and alpha in the format of dim/alpha, ie 1/1, 4/4
Training Time
Character appears to gain significant accuracy sooner than using Lora (600 steps vs 800). The time per step is a little bit slower but a better analysis of this is required. We suspect that we can train characters faster in terms of real time.
Mixing
Applying style on character appears to work well. The example wlop style applied on Amber is 1/0.25 and 4/1. The character loses some stability but the style LoCon can easily be lowered and still retain great style retention while significantly reducing the impact to character correctness. Overall it appears that mixing is more stable with LoCon than lora but this could be due to the dim/alpha and training settings between these two locon being closer compared to LoRA land where popular dim sizes range from 8 to 128.
Finetune model extractions
To be tested. We suspect the performance should be much better than LoRA extraction since all the model will be extracted from. https://github.com/KohakuBlueleaf/LoCon/blob/main/extract_locon.py