mms-1b-all-bem-genbed-m-model

This model is a fine-tuned version of facebook/mms-1b-all on the GENBED - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.2887
• Wer: 0.4119

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.6677	0.1379	100	1.1908	0.9662
0.7337	0.2759	200	0.4132	0.5658
0.5661	0.4138	300	0.3767	0.5508
0.523	0.5517	400	0.3591	0.5009
0.5391	0.6897	500	0.3530	0.5054
0.4978	0.8276	600	0.3496	0.5096
0.4737	0.9655	700	0.3478	0.5079
0.4878	1.1034	800	0.3390	0.4737
0.4737	1.2414	900	0.3304	0.4905
0.4603	1.3793	1000	0.3328	0.4842
0.4793	1.5172	1100	0.3252	0.4640
0.4359	1.6552	1200	0.3252	0.4659
0.457	1.7931	1300	0.3204	0.4717
0.469	1.9310	1400	0.3185	0.4697
0.4894	2.0690	1500	0.3167	0.4622
0.4386	2.2069	1600	0.3185	0.4646
0.4441	2.3448	1700	0.3099	0.4607
0.444	2.4828	1800	0.3154	0.4551
0.4065	2.6207	1900	0.3138	0.4623
0.4163	2.7586	2000	0.3087	0.4377
0.4518	2.8966	2100	0.3054	0.4495
0.4208	3.0345	2200	0.3037	0.4512
0.381	3.1724	2300	0.3074	0.4386
0.4203	3.3103	2400	0.2989	0.4244
0.4556	3.4483	2500	0.3080	0.4835
0.4143	3.5862	2600	0.2956	0.4222
0.4055	3.7241	2700	0.3023	0.4581
0.4102	3.8621	2800	0.2955	0.4412
0.4451	4.0	2900	0.2944	0.4181
0.3857	4.1379	3000	0.2985	0.4426
0.4071	4.2759	3100	0.2918	0.4290
0.4	4.4138	3200	0.2951	0.4321
0.4257	4.5517	3300	0.3035	0.4608
0.3929	4.6897	3400	0.2965	0.4131
0.3957	4.8276	3500	0.2938	0.4397
0.3974	4.9655	3600	0.2887	0.4119
0.3733	5.1034	3700	0.2890	0.4050
0.382	5.2414	3800	0.2917	0.4233
0.3669	5.3793	3900	0.2900	0.4313

Framework versions

• Transformers 4.46.0.dev0
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.0