samzirbo/mt5.balanced.old

mt5.balanced

This model is a fine-tuned version of samzirbo/mT5.en-es.pretrained on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4858
• Bleu: 39.6397
• Meteor: 0.6687
• Chrf++: 61.2886

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 1000
• training_steps: 30000

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Meteor	Chrf++
4.0381	0.5398	3000	2.0786	30.3181	0.59	53.6607
2.3046	1.0795	6000	1.8303	34.3855	0.6263	57.2186
2.0531	1.6193	9000	1.6969	36.2615	0.643	58.7366
1.9175	2.1591	12000	1.6209	37.2847	0.6507	59.5299
1.814	2.6988	15000	1.5698	38.2556	0.6599	60.3273
1.746	3.2386	18000	1.5331	38.7361	0.6629	60.6706
1.6921	3.7783	21000	1.5058	39.1633	0.6658	60.9563
1.6484	4.3181	24000	1.4933	39.4633	0.6677	61.1991
1.6291	4.8579	27000	1.4869	39.5943	0.6687	61.2864
1.6185	5.3976	30000	1.4858	39.6397	0.6687	61.2886

Framework versions

• Transformers 4.40.1
• Pytorch 2.2.1+cu121
• Datasets 2.19.0
• Tokenizers 0.19.1