finetune-newwiki-summarization-ver2

This model is a fine-tuned version of minnehwg/finetune-newwiki-summarization-ver1 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4697
• Rouge1: 48.1659
• Rouge2: 25.1491
• Rougel: 34.7794
• Rougelsum: 37.0893

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 7
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
0.4912	1.0	990	0.4701	48.1754	25.0221	34.7613	37.0734
0.4748	2.0	1980	0.4694	48.3629	25.3649	35.0239	37.3084
0.4755	3.0	2970	0.4695	48.2770	25.1907	34.8456	37.1930
0.4703	4.0	3960	0.4696	48.1801	25.1769	34.8004	37.0817
0.468	5.0	4950	0.4697	48.1659	25.1491	34.7794	37.0893

Framework versions

• Transformers 4.17.0
• Pytorch 2.1.2
• Datasets 2.18.0
• Tokenizers 0.15.2