bambara-mms-5-hours-mixed-asr-hf

This model is a fine-tuned version of facebook/mms-1b-all on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2515
• Wer: 0.5422
• Cer: 0.2520

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.9023	1.6750	500	1.3385	0.8473	0.3992
1.404	3.3501	1000	1.3485	0.7641	0.3617
1.2515	5.0251	1500	1.2069	0.7155	0.3408
1.1487	6.7002	2000	1.1615	0.7022	0.3248
1.0495	8.3752	2500	1.1723	0.6488	0.3015
0.9528	10.0503	3000	1.2085	0.6272	0.2902
0.8446	11.7253	3500	1.1891	0.6240	0.2888
0.7752	13.4003	4000	1.3426	0.6039	0.2792
0.7026	15.0754	4500	1.3062	0.5994	0.2773
0.6321	16.7504	5000	1.3431	0.5861	0.2724
0.5651	18.4255	5500	1.3799	0.5908	0.2733
0.5106	20.1005	6000	1.4988	0.5843	0.2722
0.4645	21.7755	6500	1.5005	0.5898	0.2738
0.4139	23.4506	7000	1.5369	0.5781	0.2705
0.378	25.1256	7500	1.6367	0.5721	0.2670
0.3389	26.8007	8000	1.7074	0.5730	0.2702
0.3067	28.4757	8500	1.7832	0.5681	0.2652
0.2842	30.1508	9000	1.8441	0.5589	0.2612
0.2576	31.8258	9500	1.7606	0.5623	0.2637
0.2333	33.5008	10000	1.8455	0.5606	0.2627
0.2192	35.1759	10500	1.9153	0.5562	0.2611
0.1992	36.8509	11000	1.8999	0.5562	0.2590
0.1802	38.5260	11500	2.0523	0.5571	0.2607
0.165	40.2010	12000	2.0747	0.5509	0.2573
0.1571	41.8760	12500	2.0351	0.5508	0.2549
0.1409	43.5511	13000	2.2121	0.5483	0.2554
0.131	45.2261	13500	2.1694	0.5458	0.2544
0.1223	46.9012	14000	2.2376	0.5455	0.2531
0.1177	48.5762	14500	2.2515	0.5422	0.2520

Framework versions

• Transformers 4.45.1
• Pytorch 2.1.0+cu118
• Datasets 2.17.0
• Tokenizers 0.20.3