pegasus-xsum-finetuned-paws-parasci

This model is a fine-tuned version of domenicrosati/pegasus-xsum-finetuned-paws on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.2256
• Rouge1: 61.8854
• Rouge2: 43.1061
• Rougel: 57.421
• Rougelsum: 57.4417

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 4000
• mixed_precision_training: Native AMP
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
No log	0.05	1000	3.8024	49.471	24.8024	43.4857	43.5552
No log	0.09	2000	3.6533	49.1046	24.4038	43.0189	43.002
No log	0.14	3000	3.5867	49.5026	24.748	43.3059	43.2923
No log	0.19	4000	3.5613	49.4319	24.5444	43.2225	43.1965

Framework versions

• Transformers 4.18.0
• Pytorch 1.11.0
• Datasets 2.1.0
• Tokenizers 0.12.1