Selimx2001x/AraT5-Arabic-To-Sign-Language-Translation

my_awesome_model

This model is a fine-tuned version of PRAli22/arat5-arabic-dialects-translation on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0131
• Bleu: 97.9438

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu
No log	1.0	50	4.6250	52.3906
No log	2.0	100	0.5771	66.2019
No log	3.0	150	0.1341	77.5175
No log	4.0	200	0.0740	87.7725
No log	5.0	250	0.0518	90.5727
No log	6.0	300	0.0372	92.5823
No log	7.0	350	0.0298	94.3032
No log	8.0	400	0.0252	95.3759
No log	9.0	450	0.0218	96.2749
1.3109	10.0	500	0.0191	96.4118
1.3109	11.0	550	0.0166	97.1165
1.3109	12.0	600	0.0149	98.0447
1.3109	13.0	650	0.0139	97.8950
1.3109	14.0	700	0.0134	97.8386
1.3109	15.0	750	0.0131	97.9438

Framework versions

• Transformers 4.37.0
• Pytorch 2.1.2
• Datasets 2.1.0
• Tokenizers 0.15.1