ghc-mistralai-Mistral-7B-v0.1-inter-frequency-model-pairwise-mse-cycle1

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6577

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss
2.4033	0.05	32	2.3607
2.2284	1.05	64	2.2275
2.0583	2.05	96	2.0617
1.8557	3.05	128	1.8781
1.579	4.05	160	1.6958
1.3837	5.05	192	1.5340
1.2205	6.05	224	1.3910
1.0612	7.05	256	1.2568
0.9495	8.05	288	1.1411
0.8527	9.05	320	1.0380
0.7652	10.05	352	0.9505
0.7067	11.05	384	0.8764
0.6501	12.05	416	0.8195
0.5988	13.05	448	0.7770
0.573	14.05	480	0.7367
0.5406	15.05	512	0.7069
0.5224	16.05	544	0.6854
0.5033	17.05	576	0.6721
0.4946	18.05	608	0.6628
0.4983	19.05	640	0.6588
0.4938	20.05	672	0.6577
0.5011	21.05	704	0.6577
0.4916	22.05	736	0.6577
0.485	23.05	768	0.6577
0.4859	24.05	800	0.6577
0.48	25.05	832	0.6577
0.4871	26.05	864	0.6577
0.4858	27.05	896	0.6577
0.4897	28.05	928	0.6577
0.4931	29.05	960	0.6577
0.4844	30.05	992	0.6577
0.4948	31.05	1024	0.6577

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.1+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0