ai-maker-space/mistral-7binstruct-summary-100s

README.md

@@ -20,7 +20,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) on the generator dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.5021
+- Loss: 1.4691
 
 ## Model description
 
@@ -52,14 +52,14 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 1.7359        | 0.22  | 25   | 1.5876          |
-| 1.5923        | 0.44  | 50   | 1.5021          |
+| 1.6719        | 0.22  | 25   | 1.5610          |
+| 1.606         | 0.43  | 50   | 1.4691          |
 
 
 ### Framework versions
 
-- PEFT 0.9.0
-- Transformers 4.38.2
-- Pytorch 2.1.0+cu121
+- PEFT 0.10.0
+- Transformers 4.39.3
+- Pytorch 2.2.1+cu121
 - Datasets 2.18.0
 - Tokenizers 0.15.2

adapter_config.json

@@ -6,6 +6,7 @@
   "fan_in_fan_out": false,
   "inference_mode": true,
   "init_lora_weights": true,
+  "layer_replication": null,
   "layers_pattern": null,
   "layers_to_transform": null,
   "loftq_config": {},
@@ -19,8 +20,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
-    "v_proj"
+    "v_proj",
+    "q_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:19a167e8ca15c107ffb36a76cdc43e6e51150422b82afb9e1727495cb921dcca
+oid sha256:891e2812c2fd9de6d594171c54bfc53f69ddd140dbe635a28b674bf35af38eaf
 size 27280152

runs/Apr16_23-05-29_a25da103e7e3/events.out.tfevents.1713308744.a25da103e7e3.1286.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7cb1a1ca965eecc7a68644343a70ce2e266daa29d03849dfe7137e3e0767454
+size 7037

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e0dc774283bf4573fc6b3950d244618c581f936c4f40bc8f914df347147d6acd
+oid sha256:63f19163dacf103e331c9475667c5e41dabe5b107dfde517c43ac274d0896b5a
 size 4920