|
--- |
|
license: apache-2.0 |
|
base_model: facebook/wav2vec2-large-xlsr-53 |
|
tags: |
|
- generated_from_trainer |
|
metrics: |
|
- wer |
|
model-index: |
|
- name: wav2vec2-large-xlsr-georgian_v1 |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# wav2vec2-large-xlsr-georgian_v1 |
|
|
|
This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the None dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.1085 |
|
- Wer: 0.2807 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 0.0003 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 16 |
|
- seed: 42 |
|
- gradient_accumulation_steps: 4 |
|
- total_train_batch_size: 64 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 200 |
|
- num_epochs: 20 |
|
- mixed_precision_training: Native AMP |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Wer | |
|
|:-------------:|:-----:|:-----:|:---------------:|:------:| |
|
| 3.1412 | 0.16 | 100 | 3.0855 | 1.0 | |
|
| 3.0569 | 0.33 | 200 | 3.0369 | 1.0 | |
|
| 2.9625 | 0.49 | 300 | 2.9778 | 1.0 | |
|
| 0.7715 | 0.65 | 400 | 0.5113 | 0.7185 | |
|
| 0.4725 | 0.81 | 500 | 0.3072 | 0.5138 | |
|
| 0.4103 | 0.98 | 600 | 0.2447 | 0.4337 | |
|
| 0.2775 | 1.14 | 700 | 0.2055 | 0.3769 | |
|
| 0.2554 | 1.3 | 800 | 0.1950 | 0.3603 | |
|
| 0.263 | 1.46 | 900 | 0.1813 | 0.3372 | |
|
| 0.2294 | 1.63 | 1000 | 0.1664 | 0.3132 | |
|
| 0.2296 | 1.79 | 1100 | 0.1565 | 0.2962 | |
|
| 0.2183 | 1.95 | 1200 | 0.1474 | 0.2986 | |
|
| 0.1822 | 2.12 | 1300 | 0.1546 | 0.2811 | |
|
| 0.1798 | 2.28 | 1400 | 0.1442 | 0.2811 | |
|
| 0.179 | 2.44 | 1500 | 0.1411 | 0.2686 | |
|
| 0.1593 | 2.6 | 1600 | 0.1408 | 0.2739 | |
|
| 0.2652 | 2.77 | 1700 | 0.2074 | 0.4499 | |
|
| 0.1834 | 2.93 | 1800 | 0.1570 | 0.3942 | |
|
| 0.2015 | 3.09 | 1900 | 0.1516 | 0.3859 | |
|
| 0.1696 | 3.26 | 2000 | 0.1452 | 0.3826 | |
|
| 0.1782 | 3.42 | 2100 | 0.1413 | 0.3763 | |
|
| 0.1636 | 3.58 | 2200 | 0.1350 | 0.3761 | |
|
| 0.173 | 3.74 | 2300 | 0.1323 | 0.3622 | |
|
| 0.1704 | 3.91 | 2400 | 0.1289 | 0.3644 | |
|
| 0.1418 | 4.07 | 2500 | 0.1266 | 0.3481 | |
|
| 0.1403 | 4.23 | 2600 | 0.1274 | 0.3482 | |
|
| 0.1353 | 4.4 | 2700 | 0.1287 | 0.3489 | |
|
| 0.1432 | 4.56 | 2800 | 0.1293 | 0.3532 | |
|
| 0.1283 | 4.72 | 2900 | 0.1226 | 0.3416 | |
|
| 0.1367 | 4.88 | 3000 | 0.1206 | 0.3426 | |
|
| 0.1162 | 5.05 | 3100 | 0.1222 | 0.3394 | |
|
| 0.1267 | 5.21 | 3200 | 0.1183 | 0.3313 | |
|
| 0.1126 | 5.37 | 3300 | 0.1180 | 0.3299 | |
|
| 0.1127 | 5.53 | 3400 | 0.1177 | 0.3305 | |
|
| 0.1155 | 5.7 | 3500 | 0.1185 | 0.3317 | |
|
| 0.1086 | 5.86 | 3600 | 0.1129 | 0.3227 | |
|
| 0.1135 | 6.02 | 3700 | 0.1118 | 0.3266 | |
|
| 0.1112 | 6.19 | 3800 | 0.1142 | 0.3228 | |
|
| 0.0866 | 6.35 | 3900 | 0.1172 | 0.3284 | |
|
| 0.1003 | 6.51 | 4000 | 0.1133 | 0.3244 | |
|
| 0.4366 | 6.68 | 4100 | 0.2436 | 0.4587 | |
|
| 0.1216 | 6.84 | 4200 | 0.1344 | 0.3386 | |
|
| 0.1165 | 7.0 | 4300 | 0.1280 | 0.3324 | |
|
| 0.131 | 7.17 | 4400 | 0.1252 | 0.3245 | |
|
| 0.1407 | 7.33 | 4500 | 0.1234 | 0.3252 | |
|
| 0.1394 | 7.49 | 4600 | 0.1208 | 0.3177 | |
|
| 0.1449 | 7.65 | 4700 | 0.1180 | 0.3165 | |
|
| 0.1295 | 7.82 | 4800 | 0.1170 | 0.3152 | |
|
| 0.1228 | 7.98 | 4900 | 0.1182 | 0.3160 | |
|
| 0.0913 | 8.14 | 5000 | 0.1122 | 0.3086 | |
|
| 0.1014 | 8.3 | 5100 | 0.1118 | 0.3100 | |
|
| 0.0861 | 8.47 | 5200 | 0.1126 | 0.3074 | |
|
| 0.1442 | 8.63 | 5300 | 0.1373 | 0.3311 | |
|
| 0.1054 | 8.79 | 5400 | 0.1225 | 0.3143 | |
|
| 0.104 | 8.96 | 5500 | 0.1190 | 0.3157 | |
|
| 0.0972 | 9.12 | 5600 | 0.1140 | 0.3076 | |
|
| 0.0948 | 9.28 | 5700 | 0.1090 | 0.3067 | |
|
| 0.1067 | 9.45 | 5800 | 0.1117 | 0.3074 | |
|
| 0.0798 | 9.61 | 5900 | 0.1097 | 0.3040 | |
|
| 0.089 | 9.77 | 6000 | 0.1049 | 0.3005 | |
|
| 0.0829 | 9.93 | 6100 | 0.1056 | 0.3006 | |
|
| 0.0687 | 10.1 | 6200 | 0.1102 | 0.3018 | |
|
| 0.0844 | 10.26 | 6300 | 0.1056 | 0.2985 | |
|
| 0.0862 | 10.42 | 6400 | 0.1073 | 0.2990 | |
|
| 0.0936 | 10.58 | 6500 | 0.1049 | 0.2949 | |
|
| 0.0821 | 10.75 | 6600 | 0.1053 | 0.2966 | |
|
| 0.0797 | 10.91 | 6700 | 0.1043 | 0.2939 | |
|
| 0.0802 | 11.07 | 6800 | 0.1057 | 0.2911 | |
|
| 0.0895 | 11.24 | 6900 | 0.1029 | 0.2934 | |
|
| 0.073 | 11.4 | 7000 | 0.1042 | 0.2897 | |
|
| 0.0842 | 11.56 | 7100 | 0.1023 | 0.2902 | |
|
| 0.0825 | 11.72 | 7200 | 0.1024 | 0.2911 | |
|
| 0.0958 | 11.89 | 7300 | 0.1018 | 0.2888 | |
|
| 0.0698 | 12.05 | 7400 | 0.1030 | 0.2883 | |
|
| 0.0693 | 12.21 | 7500 | 0.1019 | 0.2872 | |
|
| 0.0736 | 12.37 | 7600 | 0.1003 | 0.2871 | |
|
| 0.0683 | 12.54 | 7700 | 0.1004 | 0.2865 | |
|
| 0.0649 | 12.7 | 7800 | 0.1005 | 0.2835 | |
|
| 0.0669 | 12.86 | 7900 | 0.0985 | 0.2846 | |
|
| 0.069 | 13.03 | 8000 | 0.0999 | 0.2844 | |
|
| 0.0674 | 13.19 | 8100 | 0.1002 | 0.2835 | |
|
| 0.0695 | 13.35 | 8200 | 0.1013 | 0.2829 | |
|
| 0.0578 | 13.51 | 8300 | 0.1019 | 0.2821 | |
|
| 0.0614 | 13.68 | 8400 | 0.0978 | 0.2815 | |
|
| 0.0554 | 13.84 | 8500 | 0.0984 | 0.2813 | |
|
| 0.0763 | 14.0 | 8600 | 0.1001 | 0.2813 | |
|
| 0.0877 | 14.16 | 8700 | 0.1000 | 0.2808 | |
|
| 0.0882 | 14.33 | 8800 | 0.0979 | 0.2803 | |
|
| 0.0864 | 14.49 | 8900 | 0.0981 | 0.2788 | |
|
| 0.0828 | 14.65 | 9000 | 0.0975 | 0.2790 | |
|
| 0.3052 | 14.82 | 9100 | 0.2150 | 0.4175 | |
|
| 0.1478 | 14.98 | 9200 | 0.1325 | 0.3027 | |
|
| 1.0386 | 15.15 | 9300 | 0.4375 | 0.6793 | |
|
| 0.116 | 15.31 | 9400 | 0.1266 | 0.3042 | |
|
| 0.1226 | 15.47 | 9500 | 0.1206 | 0.3000 | |
|
| 0.0885 | 15.63 | 9600 | 0.1173 | 0.2958 | |
|
| 0.091 | 15.8 | 9700 | 0.1145 | 0.2929 | |
|
| 0.0886 | 15.96 | 9800 | 0.1112 | 0.2908 | |
|
| 0.0783 | 16.12 | 9900 | 0.1075 | 0.2873 | |
|
| 0.069 | 16.28 | 10000 | 0.1072 | 0.2876 | |
|
| 0.0783 | 16.45 | 10100 | 0.1070 | 0.2876 | |
|
| 0.0669 | 16.61 | 10200 | 0.1055 | 0.2848 | |
|
| 0.072 | 16.77 | 10300 | 0.1043 | 0.2846 | |
|
| 0.0721 | 16.94 | 10400 | 0.1020 | 0.2821 | |
|
| 0.0694 | 17.1 | 10500 | 0.1047 | 0.2803 | |
|
| 0.0574 | 17.26 | 10600 | 0.1053 | 0.2830 | |
|
| 0.0578 | 17.42 | 10700 | 0.1042 | 0.2806 | |
|
| 0.0663 | 17.59 | 10800 | 0.1035 | 0.2801 | |
|
| 0.0615 | 17.75 | 10900 | 0.1025 | 0.2785 | |
|
| 0.0706 | 17.91 | 11000 | 0.1028 | 0.2792 | |
|
| 0.2373 | 18.08 | 11100 | 0.1686 | 0.3372 | |
|
| 0.1137 | 18.24 | 11200 | 0.1202 | 0.2938 | |
|
| 0.1008 | 18.4 | 11300 | 0.1143 | 0.2895 | |
|
| 0.1004 | 18.57 | 11400 | 0.1127 | 0.2874 | |
|
| 0.0874 | 18.73 | 11500 | 0.1108 | 0.2861 | |
|
| 0.0926 | 18.89 | 11600 | 0.1108 | 0.2838 | |
|
| 0.0703 | 19.05 | 11700 | 0.1101 | 0.2834 | |
|
| 0.0893 | 19.22 | 11800 | 0.1097 | 0.2824 | |
|
| 0.0681 | 19.38 | 11900 | 0.1099 | 0.2822 | |
|
| 0.0668 | 19.54 | 12000 | 0.1086 | 0.2813 | |
|
| 0.069 | 19.7 | 12100 | 0.1087 | 0.2810 | |
|
| 0.0683 | 19.87 | 12200 | 0.1085 | 0.2807 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.35.2 |
|
- Pytorch 2.1.0+cu118 |
|
- Datasets 2.15.0 |
|
- Tokenizers 0.15.0 |
|
|
