my_awesome_opus_books_model

This model is a fine-tuned version of google-t5/t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.0125
• Bleu: 2.6796
• Gen Len: 16.5723

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
4.0572	1.0	875	3.5403	0.6761	16.6389
3.7186	2.0	1750	3.4207	1.0808	16.63
3.6028	3.0	2625	3.3393	1.6106	16.6017
3.5238	4.0	3500	3.2808	1.8017	16.5734
3.4853	5.0	4375	3.2328	1.8614	16.5654
3.4414	6.0	5250	3.1941	2.0524	16.5806
3.3865	7.0	6125	3.1596	2.066	16.5949
3.3489	8.0	7000	3.1347	2.209	16.5674
3.3318	9.0	7875	3.1102	2.2722	16.582
3.314	10.0	8750	3.0918	2.361	16.5643
3.2818	11.0	9625	3.0754	2.4055	16.5391
3.2743	12.0	10500	3.0600	2.4443	16.5766
3.2624	13.0	11375	3.0497	2.5144	16.5469
3.2403	14.0	12250	3.0389	2.5359	16.5914
3.2363	15.0	13125	3.0311	2.6375	16.5534
3.2109	16.0	14000	3.0243	2.6004	16.556
3.2168	17.0	14875	3.0197	2.6429	16.5517
3.1997	18.0	15750	3.0148	2.6728	16.5611
3.1963	19.0	16625	3.0131	2.6555	16.58
3.1958	20.0	17500	3.0125	2.6796	16.5723

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.0+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3