LifeScienceBARTPrincipal

This model is a fine-tuned version of facebook/bart-large-cnn on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 4.6738
• Rouge1: 49.5951
• Rouge2: 14.7925
• Rougel: 33.5728
• Rougelsum: 46.0607
• Bertscore Precision: 81.3188
• Bertscore Recall: 83.0508
• Bertscore F1: 82.1725
• Bleu: 0.1002
• Gen Len: 227.8658

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bertscore Precision	Bertscore Recall	Bertscore F1	Bleu	Gen Len
6.522	0.0881	100	6.3500	39.9657	10.1487	26.5496	37.2024	77.4604	80.5886	78.9876	0.0657	227.8658
6.0286	0.1762	200	5.9070	38.2642	10.0835	26.309	35.3774	76.6233	80.7328	78.6174	0.0677	227.8658
5.633	0.2643	300	5.5510	46.5639	12.278	29.4476	43.4837	79.3971	81.6021	80.4809	0.0788	227.8658
5.4363	0.3524	400	5.3379	45.7917	12.3778	29.8157	42.5627	79.5999	81.8994	80.7293	0.0818	227.8658
5.3556	0.4405	500	5.1825	45.7896	12.7724	30.8163	42.7893	80.125	82.0725	81.0835	0.0839	227.8658
5.176	0.5286	600	5.0198	46.4704	13.3572	31.4246	43.3941	80.2167	82.2861	81.2343	0.0877	227.8658
5.0712	0.6167	700	4.9271	49.2365	14.1585	32.4079	45.6186	81.1084	82.7557	81.9207	0.0939	227.8658
4.9175	0.7048	800	4.8257	48.3333	14.04	32.7436	44.7264	80.8876	82.7874	81.8231	0.0947	227.8658
4.9291	0.7929	900	4.7636	49.1012	14.4773	33.113	45.5053	80.9593	82.9139	81.9214	0.0977	227.8658
4.6748	0.8810	1000	4.7083	49.9025	14.8866	33.4691	46.352	81.3587	83.0297	82.1826	0.1002	227.8658
4.8064	0.9691	1100	4.6738	49.5951	14.7925	33.5728	46.0607	81.3188	83.0508	82.1725	0.1002	227.8658

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1