S3nh's Axolotl Finetuned
Collection
Collection of LLMs finetuned using axolotl library, mostly
•
5 items
•
Updated
This model is a fine-tuned version of microsoft/phi-2 on the None dataset. It achieves the following results on the evaluation set:
More information needed
More information needed
More information needed
The following hyperparameters were used during training:
Training Loss | Epoch | Step | Validation Loss |
---|---|---|---|
No log | 0.0 | 1 | 1.7973 |
1.9767 | 0.25 | 5290 | 1.4832 |
1.8474 | 0.5 | 10580 | 1.4356 |
1.8121 | 0.75 | 15870 | 1.4022 |
1.8333 | 1.0 | 21160 | 1.3678 |
1.6601 | 1.25 | 26450 | 1.3508 |
1.5452 | 1.5 | 31740 | 1.3357 |
1.7381 | 1.75 | 37030 | 1.3191 |
1.6256 | 2.0 | 42320 | 1.3090 |
1.5521 | 2.25 | 47610 | 1.2961 |
1.8318 | 2.5 | 52900 | 1.2910 |
1.6761 | 2.75 | 58190 | 1.2901 |
1.6312 | 3.0 | 63480 | 1.2879 |
1.7003 | 3.25 | 68770 | 1.2820 |
1.6915 | 3.5 | 74060 | 1.2814 |
1.5757 | 3.75 | 79350 | 1.2813 |
The following bitsandbytes
quantization config was used during training: