README.md

metadata

base_model: UBC-NLP/AraT5v2-base-1024
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: results_arat5_wiki
    results: []

results_arat5_wiki

This model is a fine-tuned version of UBC-NLP/AraT5v2-base-1024 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.4401
• Rouge1: 0.0905
• Rouge2: 0.0
• Rougel: 0.0915
• Rougelsum: 0.0912
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 250
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
7.7921	0.4757	500	6.2870	0.0905	0.0	0.0915	0.0912	19.0
5.9839	0.9515	1000	5.5934	0.0905	0.0	0.0915	0.0912	19.0
5.4311	1.4272	1500	5.0896	0.0905	0.0	0.0915	0.0912	19.0
5.1245	1.9029	2000	4.7004	0.0905	0.0	0.0915	0.0912	19.0
4.7258	2.3787	2500	4.3347	0.0905	0.0	0.0915	0.0912	19.0
4.5072	2.8544	3000	4.0503	0.0905	0.0	0.0915	0.0912	19.0
4.2388	3.3302	3500	3.8321	0.0905	0.0	0.0915	0.0912	19.0
4.0817	3.8059	4000	3.6509	0.0905	0.0	0.0915	0.0912	19.0
3.8843	4.2816	4500	3.4451	0.0905	0.0	0.0915	0.0912	19.0
3.7958	4.7574	5000	3.3071	0.0905	0.0	0.0915	0.0912	19.0
3.6397	5.2331	5500	3.1619	0.0905	0.0	0.0915	0.0912	19.0
3.5658	5.7088	6000	3.0068	0.0905	0.0	0.0915	0.0912	19.0
3.4171	6.1846	6500	2.9459	0.0905	0.0	0.0915	0.0912	19.0
3.2697	6.6603	7000	2.8074	0.0842	0.0	0.0849	0.0844	19.0
3.3168	7.1361	7500	2.7153	0.0905	0.0	0.0915	0.0912	19.0
3.1594	7.6118	8000	2.6676	0.0905	0.0	0.0915	0.0912	19.0
3.0928	8.0875	8500	2.5849	0.0905	0.0	0.0915	0.0912	19.0
3.0318	8.5633	9000	2.5152	0.0905	0.0	0.0915	0.0912	19.0
3.0392	9.0390	9500	2.4849	0.0902	0.0	0.0911	0.0908	19.0
2.9917	9.5147	10000	2.4569	0.0768	0.0001	0.0774	0.0768	19.0
2.9281	9.9905	10500	2.4401	0.0905	0.0	0.0915	0.0912	19.0

Framework versions

• Transformers 4.42.0.dev0
• Pytorch 2.2.1+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1