wav2vec2-xls-r-1b-faroese-100h-10k-steps

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.4747
• Wer: 100.0
• Cer: 82.7552

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
2.0903	0.4640	1000	0.6799	73.2837	24.4656
2.872	0.9281	2000	0.9368	81.5331	31.3250
2.9863	1.3921	3000	0.8913	79.8873	29.0572
2.5887	1.8561	4000	0.9178	81.5633	30.7605
2.5408	2.3202	5000	1.0047	85.0765	35.1896
3.057	2.7842	6000	1.0349	82.6958	32.6409
6.9153	3.2483	7000	2.9900	100.0	98.8214
24.2761	3.7123	8000	11.5877	100.0	98.8530
5.0923	4.1763	9000	2.3517	100.0	90.6749
5.2345	4.6404	10000	2.4747	100.0	82.7552

Framework versions

• Transformers 4.46.1
• Pytorch 2.5.0+cu121
• Datasets 3.1.0
• Tokenizers 0.20.2