xlsr-no-mo-nmcpc

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0003
• Wer: 0.2468

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0004
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 132
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
4.8398	4.1667	200	3.0698	1.0
2.9735	8.3333	400	2.6202	1.0
1.8952	12.5	600	0.5988	0.6128
0.6816	16.6667	800	0.2092	0.4191
0.3621	20.8333	1000	0.0634	0.3383
0.2277	25.0	1200	0.0534	0.3128
0.1648	29.1667	1400	0.0369	0.3021
0.1169	33.3333	1600	0.0165	0.2681
0.1101	37.5	1800	0.0124	0.2489
0.0912	41.6667	2000	0.0088	0.2553
0.066	45.8333	2200	0.0023	0.2489
0.0565	50.0	2400	0.0112	0.2553
0.0465	54.1667	2600	0.0036	0.2511
0.0444	58.3333	2800	0.0028	0.2553
0.0436	62.5	3000	0.0023	0.2511
0.0418	66.6667	3200	0.0009	0.2511
0.0299	70.8333	3400	0.0017	0.2489
0.0246	75.0	3600	0.0006	0.2468
0.0184	79.1667	3800	0.0010	0.2468
0.0163	83.3333	4000	0.0012	0.2532
0.0143	87.5	4200	0.0009	0.2489
0.0149	91.6667	4400	0.0004	0.2468
0.0125	95.8333	4600	0.0003	0.2468
0.0106	100.0	4800	0.0003	0.2468

Framework versions

• Transformers 4.45.0.dev0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1