---
library_name: transformers
license: cc-by-nc-4.0
base_model: facebook/mms-1b-all
tags:
- generated_from_trainer
datasets:
- common_voice_17_0
metrics:
- wer
- bleu
model-index:
- name: wav2vec2-mms-1b-CV17.0-training_set_variations
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: common_voice_17_0
      type: common_voice_17_0
      config: ta
      split: validation
      args: ta
    metrics:
    - name: Wer
      type: wer
      value: 1.0759957831049183
    - name: Bleu
      type: bleu
      value: 0.0
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-mms-1b-CV17.0-training_set_variations

This model is a fine-tuned version of [facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all) on the common_voice_17_0 dataset.
It achieves the following results on the evaluation set:
- Loss: 6.3100
- Wer: 1.0760
- Cer: 0.7242
- Bleu: 0.0

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.15
- training_steps: 2000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Wer    | Cer    | Bleu |
|:-------------:|:------:|:----:|:---------------:|:------:|:------:|:----:|
| 13.5071       | 6.25   | 50   | 7.0048          | 1.0001 | 0.9823 | 0.0  |
| 5.6893        | 12.5   | 100  | 5.2167          | 1.0000 | 0.9089 | 0.0  |
| 4.3569        | 18.75  | 150  | 4.3516          | 1.0125 | 0.8661 | 0.0  |
| 3.5211        | 25.0   | 200  | 3.5506          | 1.0269 | 0.8318 | 0.0  |
| 3.2359        | 31.25  | 250  | 3.4822          | 1.0039 | 0.8458 | 0.0  |
| 3.1004        | 37.5   | 300  | 3.5209          | 1.0086 | 0.8185 | 0.0  |
| 2.9953        | 43.75  | 350  | 3.5498          | 1.0121 | 0.8104 | 0.0  |
| 2.779         | 50.0   | 400  | 3.6218          | 1.0170 | 0.7777 | 0.0  |
| 2.6019        | 56.25  | 450  | 4.0907          | 1.0134 | 0.7504 | 0.0  |
| 2.4589        | 62.5   | 500  | 3.8633          | 1.0287 | 0.7476 | 0.0  |
| 2.2318        | 68.75  | 550  | 3.7976          | 1.0367 | 0.7239 | 0.0  |
| 2.0073        | 75.0   | 600  | 4.0050          | 1.0234 | 0.7288 | 0.0  |
| 1.7416        | 81.25  | 650  | 4.2022          | 1.0126 | 0.7231 | 0.0  |
| 1.5467        | 87.5   | 700  | 4.4087          | 1.0469 | 0.7197 | 0.0  |
| 1.3716        | 93.75  | 750  | 4.5391          | 1.0471 | 0.7185 | 0.0  |
| 1.2237        | 100.0  | 800  | 4.8405          | 1.0398 | 0.7152 | 0.0  |
| 1.1216        | 106.25 | 850  | 5.0209          | 1.0421 | 0.7160 | 0.0  |
| 1.0274        | 112.5  | 900  | 5.0349          | 1.0669 | 0.7160 | 0.0  |
| 0.9325        | 118.75 | 950  | 5.2384          | 1.0500 | 0.7198 | 0.0  |
| 0.8486        | 125.0  | 1000 | 5.3843          | 1.0614 | 0.7131 | 0.0  |
| 0.7996        | 131.25 | 1050 | 5.4558          | 1.0622 | 0.7181 | 0.0  |
| 0.7479        | 137.5  | 1100 | 5.6746          | 1.0622 | 0.7205 | 0.0  |
| 0.6972        | 143.75 | 1150 | 5.7069          | 1.1182 | 0.7218 | 0.0  |
| 0.6596        | 150.0  | 1200 | 5.7678          | 1.0883 | 0.7221 | 0.0  |
| 0.6213        | 156.25 | 1250 | 5.9645          | 1.0700 | 0.7191 | 0.0  |
| 0.5905        | 162.5  | 1300 | 6.0098          | 1.0970 | 0.7210 | 0.0  |
| 0.5532        | 168.75 | 1350 | 6.0379          | 1.0981 | 0.7235 | 0.0  |
| 0.5336        | 175.0  | 1400 | 6.2079          | 1.0564 | 0.7210 | 0.0  |
| 0.4955        | 181.25 | 1450 | 6.1618          | 1.0717 | 0.7225 | 0.0  |
| 0.4851        | 187.5  | 1500 | 6.3100          | 1.0760 | 0.7242 | 0.0  |


### Framework versions

- Transformers 4.44.2
- Pytorch 2.4.1+cu121
- Datasets 3.0.0
- Tokenizers 0.19.1