t5-small-t5-dialogue-summarizer

This model is a fine-tuned version of t5-small on the samsum dataset. It achieves the following results on the evaluation set:

• Loss: 1.7032
• Rouge1: 43.3371
• Rouge2: 20.6294
• Rougel: 36.6607
• Rougelsum: 40.209
• Gen Len: 16.698

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	460	1.8115	41.2589	18.3552	34.5107	38.2488	16.8068
1.9846	2.0	921	1.7892	41.1617	18.4345	34.745	38.2061	16.6247
1.9568	3.0	1381	1.7757	41.7317	19.0104	35.2965	38.6958	16.4059
1.9298	4.0	1842	1.7573	42.0478	19.1229	35.4855	39.0882	16.6235
1.9049	5.0	2302	1.7496	42.4985	19.5594	35.9228	39.4201	16.5416
1.8852	6.0	2763	1.7411	42.3214	19.6152	35.7488	39.3079	16.7139
1.8674	7.0	3223	1.7335	42.3206	19.7528	35.9918	39.2783	16.5073
1.855	8.0	3684	1.7300	42.9099	20.2273	36.4393	39.8506	16.61
1.8435	9.0	4144	1.7225	42.9661	20.3074	36.3468	39.8945	16.7103
1.8342	10.0	4605	1.7198	43.0181	20.2982	36.4202	39.9022	16.7726
1.8216	11.0	5065	1.7169	43.0296	20.5422	36.6314	40.111	16.6883
1.8168	12.0	5526	1.7144	43.3035	20.7167	36.7924	40.2953	16.7787
1.8168	13.0	5986	1.7104	43.2258	20.7416	36.7823	40.2551	16.7286
1.8088	14.0	6447	1.7075	43.3982	20.8281	36.8254	40.3198	16.7384
1.8008	15.0	6907	1.7079	43.3077	20.7164	36.6791	40.2372	16.687
1.8014	16.0	7368	1.7047	43.1989	20.6984	36.7104	40.2285	16.6479
1.7934	17.0	7828	1.7034	43.4149	20.7879	36.7308	40.3556	16.7922
1.7894	18.0	8289	1.7041	43.2962	20.7667	36.7017	40.28	16.6883
1.7914	19.0	8749	1.7037	43.2489	20.6943	36.676	40.1802	16.6932
1.7827	19.98	9200	1.7032	43.3371	20.6294	36.6607	40.209	16.698

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0