model

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0530
• Rouge1: 78.2421
• Rouge2: 75.2403
• Rougel: 78.1859
• Rougelsum: 78.1938
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.0717	0.25	1000	0.0637	78.0577	74.8304	78.018	78.0236	19.0
0.0659	0.5	2000	0.0598	78.1871	75.0081	78.134	78.1354	19.0
0.0624	0.75	3000	0.0576	77.9279	74.7627	77.868	77.88	19.0
0.0597	1.01	4000	0.0563	78.1154	75.0171	78.0426	78.0522	19.0
0.0565	1.26	5000	0.0556	78.1454	75.0883	78.0991	78.1097	19.0
0.0528	1.51	6000	0.0549	78.3856	75.379	78.318	78.3263	19.0
0.0563	1.76	7000	0.0541	78.2664	75.1877	78.1834	78.1963	19.0
0.0535	2.01	8000	0.0540	78.1601	75.1107	78.0957	78.1042	19.0
0.0507	2.26	9000	0.0537	78.2119	75.137	78.1504	78.16	19.0
0.0513	2.51	10000	0.0533	78.1715	75.1571	78.1235	78.1272	19.0
0.052	2.76	11000	0.0530	78.2421	75.2403	78.1859	78.1938	19.0

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1