Training in progress, step 300

README.md

@@ -17,8 +17,8 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [ad019el/tamasheq-99-1](https://huggingface.co/ad019el/tamasheq-99-1) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.3783
-- Wer: 0.8147
+- Loss: 0.3479
+- Wer: 0.4957
 
 ## Model description
 
@@ -46,23 +46,22 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_steps: 500
-- num_epochs: 200
+- num_epochs: 30
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss | Wer    |
 |:-------------:|:-----:|:----:|:---------------:|:------:|
-| 7.9063        | 7.89  | 300  | 3.0656          | 1.0    |
-| 2.7952        | 15.79 | 600  | 1.7388          | 0.9324 |
-| 1.2354        | 23.68 | 900  | 1.0927          | 0.8618 |
-| 0.8131        | 31.58 | 1200 | 1.1919          | 0.8618 |
-| 0.6311        | 39.47 | 1500 | 1.2800          | 0.8559 |
-| 0.5422        | 47.37 | 1800 | 1.3783          | 0.8147 |
+| 4.3118        | 6.0   | 300  | 0.8477          | 0.8593 |
+| 0.4823        | 12.0  | 600  | 0.3741          | 0.5064 |
+| 0.2054        | 18.0  | 900  | 0.3855          | 0.5027 |
+| 0.1798        | 24.0  | 1200 | 0.3700          | 0.5023 |
+| 0.2097        | 30.0  | 1500 | 0.3479          | 0.4957 |
 
 
 ### Framework versions
 
-- Transformers 4.32.1
+- Transformers 4.31.0
 - Pytorch 2.0.1+cu118
 - Datasets 2.14.4
 - Tokenizers 0.13.3

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "ad019el/tamasheq-99-1",
+  "_name_or_path": "/content/tamasheq-99-2",
   "activation_dropout": 0.05,
   "adapter_attn_dim": null,
   "adapter_kernel_size": 3,
@@ -85,7 +85,7 @@
   "num_hidden_layers": 24,
   "num_negatives": 100,
   "output_hidden_size": 1024,
-  "pad_token_id": 44,
+  "pad_token_id": 43,
   "proj_codevector_dim": 256,
   "tdnn_dilation": [
     1,
@@ -111,6 +111,6 @@
   "torch_dtype": "float32",
   "transformers_version": "4.32.1",
   "use_weighted_layer_sum": false,
-  "vocab_size": 45,
+  "vocab_size": 44,
   "xvector_output_dim": 512
 }

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3c8feeefdbe72b0a41e30cab1e9d5ad58ac42c840f45f190ecf4c1b58c59fc93
-size 1262086317
+oid sha256:b4d0fcb51f2e603b102e660dea05870208f71f1eac2f418643e3652cef2e3a29
+size 1262082221

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8ecf73b417a818124b2cc32e2e272cb5eb46ecc402af6feddd6b0f42ac9d7a73
+oid sha256:ad6c7003da7fe6691dd038465b0e03ae7c241e32bc2461e2ac611d3d835b8a3c
 size 4027