DanSumT5-largeV_84227

This model is a fine-tuned version of Danish-summarisation/DanSumT5-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.2976
• Rouge1: 32.3488
• Rouge2: 8.638
• Rougel: 18.8215
• Rougelsum: 29.8654
• Gen Len: 126.28

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	200	2.5620	31.6386	7.3603	17.9932	28.8935	126.32
No log	2.0	400	2.4824	31.8478	8.0477	18.5952	29.2582	126.77
2.7655	3.0	600	2.4305	32.1965	8.4935	18.7317	29.9719	125.03
2.7655	4.0	800	2.3945	31.8539	8.7262	18.5421	29.8472	125.63
2.4368	5.0	1000	2.3685	32.0137	8.2933	18.7818	29.561	125.32
2.4368	6.0	1200	2.3522	31.5	8.3477	18.9478	29.3072	125.11
2.4368	7.0	1400	2.3364	31.6482	8.3012	18.9953	29.0985	123.38
2.2645	8.0	1600	2.3250	31.9939	8.5944	18.9914	29.5092	125.18
2.2645	9.0	1800	2.3212	31.5611	8.1969	18.7941	29.151	126.01
2.134	10.0	2000	2.3117	32.0902	8.6962	19.0793	29.758	125.4
2.134	11.0	2200	2.3064	31.9365	8.7161	18.9113	29.6812	125.86
2.134	12.0	2400	2.3062	32.3185	9.0913	19.2692	29.9962	126.24
2.0467	13.0	2600	2.3032	31.7591	8.4993	18.8326	29.4231	125.02
2.0467	14.0	2800	2.3008	32.0532	8.8654	18.897	29.5819	126.2
1.9931	15.0	3000	2.2980	31.8987	8.7669	19.0859	29.3799	126.0
1.9931	16.0	3200	2.2982	32.2458	8.7896	18.6845	29.6991	126.0
1.9931	17.0	3400	2.2987	32.0869	8.6678	18.7656	29.8441	125.66
1.949	18.0	3600	2.2974	32.1759	8.6004	18.7892	29.6918	126.31
1.949	19.0	3800	2.2970	32.1139	8.5827	18.7099	29.5327	126.15
1.9257	20.0	4000	2.2976	32.3488	8.638	18.8215	29.8654	126.28

Framework versions

• Transformers 4.32.1
• Pytorch 2.1.0
• Datasets 2.12.0
• Tokenizers 0.13.3