training after 1 epoch; batch=64

README.md

@@ -5,9 +5,6 @@ tags:
 - generated_from_trainer
 datasets:
 - emotion
-metrics:
-- accuracy
-- f1
 model-index:
 - name: llama-2-7B-Guanaco-QLoRA-AWQ
   results: []
@@ -19,10 +16,6 @@ should probably proofread and complete it, then remove this comment. -->
 # llama-2-7B-Guanaco-QLoRA-AWQ
 
 This model is a fine-tuned version of [TheBloke/llama-2-7B-Guanaco-QLoRA-AWQ](https://huggingface.co/TheBloke/llama-2-7B-Guanaco-QLoRA-AWQ) on the emotion dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.7119
-- Accuracy: 0.778
-- F1: 0.7718
 
 ## Model description
 
@@ -42,28 +35,19 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 2e-05
-- train_batch_size: 8
-- eval_batch_size: 8
+- train_batch_size: 64
+- eval_batch_size: 64
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 10
+- num_epochs: 1
 - mixed_precision_training: Native AMP
 
 ### Training results
 
-| Training Loss | Epoch | Step  | Validation Loss | Accuracy | F1     |
-|:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|
-| 1.5456        | 1.0   | 2000  | 1.5658          | 0.397    | 0.2952 |
-| 1.3418        | 2.0   | 4000  | 1.4285          | 0.483    | 0.4464 |
-| 1.1199        | 3.0   | 6000  | 1.3052          | 0.5285   | 0.4825 |
-| 0.9157        | 4.0   | 8000  | 1.1448          | 0.5925   | 0.5616 |
-| 0.695         | 5.0   | 10000 | 0.9214          | 0.6745   | 0.6638 |
-| 0.5373        | 6.0   | 12000 | 0.8784          | 0.6925   | 0.6931 |
-| 0.405         | 7.0   | 14000 | 0.7437          | 0.745    | 0.7362 |
-| 0.2908        | 8.0   | 16000 | 0.7283          | 0.7625   | 0.7538 |
-| 0.2407        | 9.0   | 18000 | 0.6977          | 0.7775   | 0.7745 |
-| 0.1836        | 10.0  | 20000 | 0.7119          | 0.778    | 0.7718 |
+| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
+| No log        | 1.0   | 250  | 1.6493          | 0.31     | 0.2471 |
 
 
 ### Framework versions

adapter_config.json

@@ -17,8 +17,6 @@
   "revision": null,
   "target_modules": [
     "v_proj",
-    "k_proj",
-    "o_proj",
     "q_proj"
   ],
   "task_type": "SEQ_CLS"

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f6ffed09dc5c9fb6fde1c64e4b5e80f2d609aad847bcf50e010347e87385265f
-size 33686928
+oid sha256:c70f4972e653b6005420e3ebb0a548f9212c77521686f2b963cd0f64a04a1b5e
+size 16892600

runs/Nov09_16-04-23_d60396118908/events.out.tfevents.1699545921.d60396118908.17900.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:abac623df8d5d9e063e53f54ae3615c6323ffb39ab21224509b47f54ba1d0337
+size 5057

runs/Nov09_16-07-16_d60396118908/events.out.tfevents.1699546053.d60396118908.17900.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f42ef147a05e6a523ad753a005aa9dbac732b8a037455d090169d7c9ff8783f
+size 5831

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d0f81dab4457613bbc2b3000a8748f0bcd6e8fa15b2960467b741767f85369b3
+oid sha256:0f609275ad152bb3c0a53731438dae398abcc128d9f596d40a5912ab702c703e
 size 4664