Model save

README.md

@@ -0,0 +1,75 @@
+---
+license: apache-2.0
+base_model: riotu-lab/ArabianGPT-01B
+tags:
+- generated_from_trainer
+metrics:
+- bleu
+- rouge
+model-index:
+- name: res_nw_yem
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# res_nw_yem
+
+This model is a fine-tuned version of [riotu-lab/ArabianGPT-01B](https://huggingface.co/riotu-lab/ArabianGPT-01B) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.4885
+- Bleu: 0.2560
+- Rouge1: 0.6116
+- Rouge2: 0.3795
+- Rougel: 0.6086
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 500
+- num_epochs: 20.0
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Bleu   | Rouge1 | Rouge2 | Rougel |
+|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:------:|
+| 7.8314        | 1.0   | 153  | 2.3597          | 0.0104 | 0.0983 | 0.0018 | 0.0964 |
+| 0.9995        | 2.0   | 306  | 0.5538          | 0.1161 | 0.4736 | 0.1854 | 0.4728 |
+| 0.4848        | 3.0   | 459  | 0.5034          | 0.1472 | 0.5008 | 0.2174 | 0.5002 |
+| 0.3823        | 4.0   | 612  | 0.4827          | 0.1912 | 0.5331 | 0.2744 | 0.5314 |
+| 0.293         | 5.0   | 765  | 0.4732          | 0.2096 | 0.5619 | 0.3099 | 0.5587 |
+| 0.2239        | 6.0   | 918  | 0.4720          | 0.2297 | 0.5777 | 0.3341 | 0.5758 |
+| 0.1766        | 7.0   | 1071 | 0.4737          | 0.2302 | 0.5885 | 0.3468 | 0.5872 |
+| 0.1434        | 8.0   | 1224 | 0.4758          | 0.2451 | 0.5938 | 0.3648 | 0.5903 |
+| 0.1202        | 9.0   | 1377 | 0.4828          | 0.2509 | 0.6047 | 0.3683 | 0.6015 |
+| 0.1042        | 10.0  | 1530 | 0.4852          | 0.2439 | 0.5980 | 0.3704 | 0.5951 |
+| 0.0955        | 11.0  | 1683 | 0.4885          | 0.2560 | 0.6116 | 0.3795 | 0.6086 |
+
+
+### Framework versions
+
+- Transformers 4.45.0.dev0
+- Pytorch 2.3.1+cu121
+- Datasets 2.19.2
+- Tokenizers 0.19.1

egy_training_log.txt

@@ -165,3 +165,8 @@ INFO:absl:Using default tokenizer.
 INFO:root:Epoch 9.0: Train Loss = 0.1434, Eval Loss = 0.47583380341529846
 INFO:absl:Using default tokenizer.
 INFO:root:Epoch 10.0: Train Loss = 0.1202, Eval Loss = 0.4827924072742462
+INFO:absl:Using default tokenizer.
+INFO:root:Epoch 11.0: Train Loss = 0.1042, Eval Loss = 0.4852388799190521
+INFO:absl:Using default tokenizer.
+WARNING:huggingface_hub.utils._http:'(MaxRetryError("HTTPSConnectionPool(host='hf-hub-lfs-us-east-1.s3-accelerate.amazonaws.com', port=443): Max retries exceeded with url: /repos/4e/c5/4ec5c12bd4b31fb218515ee480d86e26419d948a9a596a06a6ad09ce77d37e3a/7e7cfbb2940a8097a9e6e8e5bc5cd9f274cf2ba28fd92151cbe4dd0e72ae5712?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA2JU7TKAQLC2QXPN7%2F20240831%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240831T191048Z&X-Amz-Expires=86400&X-Amz-Signature=d59b69eb9175411687ff7810cd33c6c85c44bf981f49b4f8ae534408ec3562e2&X-Amz-SignedHeaders=host&partNumber=2&uploadId=xNqFvN721XV0SFXx42EP0sJDJW.W2QqIOgUvk5YQWI9SYvlnvF8kCxC_2mjzHt0k5t51hFB.oftAU6tGWfNhGNpUPE9HMn_UnOlRe.JGXG5J64HYNr3rofly6kQEo7ZO&x-id=UploadPart (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2406)')))"), '(Request ID: 784209eb-439a-4fa9-b9f8-072965b6bd5d)')' thrown while requesting PUT https://hf-hub-lfs-us-east-1.s3-accelerate.amazonaws.com/repos/4e/c5/4ec5c12bd4b31fb218515ee480d86e26419d948a9a596a06a6ad09ce77d37e3a/7e7cfbb2940a8097a9e6e8e5bc5cd9f274cf2ba28fd92151cbe4dd0e72ae5712?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA2JU7TKAQLC2QXPN7%2F20240831%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240831T191048Z&X-Amz-Expires=86400&X-Amz-Signature=d59b69eb9175411687ff7810cd33c6c85c44bf981f49b4f8ae534408ec3562e2&X-Amz-SignedHeaders=host&partNumber=2&uploadId=xNqFvN721XV0SFXx42EP0sJDJW.W2QqIOgUvk5YQWI9SYvlnvF8kCxC_2mjzHt0k5t51hFB.oftAU6tGWfNhGNpUPE9HMn_UnOlRe.JGXG5J64HYNr3rofly6kQEo7ZO&x-id=UploadPart
+WARNING:huggingface_hub.utils._http:Retrying in 1s [Retry 1/5].

generation_config.json

@@ -0,0 +1,5 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": 64000,
+  "transformers_version": "4.45.0.dev0"
+}