SBIC-mistralai-Mistral-7B-v0.1-intra-dataset-frequency-model-pairwise-mse-cycle1

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5520

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss
2.4802	0.04	31	2.4091
2.2997	1.04	62	2.2583
2.1019	2.04	93	2.0836
1.8498	3.04	124	1.8917
1.6275	4.04	155	1.7154
1.4091	5.04	186	1.5379
1.2361	6.04	217	1.3779
1.087	7.04	248	1.2654
0.9664	8.04	279	1.1340
0.8312	9.04	310	1.0311
0.7554	10.04	341	0.9393
0.6873	11.04	372	0.8599
0.6292	12.04	403	0.7976
0.5822	13.04	434	0.7410
0.5453	14.04	465	0.6989
0.5115	15.04	496	0.6611
0.4778	16.04	527	0.6292
0.4527	17.04	558	0.6054
0.4474	18.04	589	0.5882
0.4285	19.04	620	0.5740
0.4193	20.04	651	0.5642
0.4061	21.04	682	0.5583
0.4044	22.04	713	0.5544
0.4	23.04	744	0.5524
0.4029	24.04	775	0.5520
0.3973	25.04	806	0.5520
0.3977	26.04	837	0.5520
0.3947	27.04	868	0.5520
0.3983	28.04	899	0.5520
0.3934	29.04	930	0.5520
0.3977	30.04	961	0.5520
0.4022	31.04	992	0.5520

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.1+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0