wav2vec2_XLSR_darija_maroc

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2740
• Wer: 0.3247

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
4.9874	0.83	400	2.4916	0.9978
0.9314	1.66	800	0.4379	0.5839
0.4856	2.49	1200	0.3485	0.5001
0.3859	3.32	1600	0.3331	0.4768
0.3271	4.15	2000	0.2932	0.4369
0.2842	4.97	2400	0.2988	0.4422
0.2482	5.8	2800	0.2815	0.4174
0.2252	6.63	3200	0.2967	0.4316
0.2115	7.46	3600	0.2719	0.4007
0.1965	8.29	4000	0.2867	0.3985
0.1823	9.12	4400	0.2744	0.4008
0.1712	9.95	4800	0.2589	0.3873
0.156	10.78	5200	0.2589	0.3745
0.1536	11.61	5600	0.2628	0.3799
0.1404	12.44	6000	0.2704	0.3805
0.1402	13.26	6400	0.2748	0.3784
0.1305	14.09	6800	0.2780	0.3769
0.121	14.92	7200	0.2760	0.3698
0.1147	15.75	7600	0.2723	0.3733
0.1076	16.58	8000	0.2661	0.3671
0.1013	17.41	8400	0.2709	0.3665
0.0964	18.24	8800	0.2748	0.3599
0.0946	19.07	9200	0.2696	0.3550
0.0916	19.9	9600	0.2649	0.3596
0.0849	20.73	10000	0.2905	0.3573
0.0803	21.55	10400	0.2646	0.3496
0.0748	22.38	10800	0.2871	0.3486
0.0739	23.21	11200	0.2751	0.3432
0.0699	24.04	11600	0.2857	0.3426
0.0637	24.87	12000	0.2690	0.3377
0.0627	25.7	12400	0.2737	0.3371
0.0601	26.53	12800	0.2752	0.3358
0.0564	27.36	13200	0.2827	0.3329
0.0536	28.19	13600	0.2764	0.3310
0.0508	29.02	14000	0.2750	0.3281
0.0495	29.84	14400	0.2740	0.3247

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1