DANWPDO/openhermes-mistral-dpo-gptq

README.md

@@ -17,16 +17,6 @@ should probably proofread and complete it, then remove this comment. -->
 # openhermes-mistral-dpo-gptq
 
 This model is a fine-tuned version of [TheBloke/OpenHermes-2-Mistral-7B-GPTQ](https://huggingface.co/TheBloke/OpenHermes-2-Mistral-7B-GPTQ) on the None dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.6780
-- Rewards/chosen: 0.0314
-- Rewards/rejected: -0.0026
-- Rewards/accuracies: 0.6875
-- Rewards/margins: 0.0340
-- Logps/rejected: -157.6995
-- Logps/chosen: -203.0707
-- Logits/rejected: -2.3164
-- Logits/chosen: -2.4209
 
 ## Model description
 
@@ -55,20 +45,9 @@ The following hyperparameters were used during training:
 - training_steps: 50
 - mixed_precision_training: Native AMP
 
-### Training results
-
-| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
-|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
-| 0.6932        | 0.01  | 10   | 0.6910          | -0.0090        | -0.0070          | 0.25               | -0.0020         | -157.7437      | -203.4745    | -2.3168         | -2.4207       |
-| 0.6905        | 0.01  | 20   | 0.6893          | -0.0028        | -0.0049          | 0.1875             | 0.0022          | -157.7232      | -203.4128    | -2.3159         | -2.4207       |
-| 0.6802        | 0.01  | 30   | 0.6868          | 0.0121         | -0.0011          | 0.6875             | 0.0132          | -157.6847      | -203.2636    | -2.3163         | -2.4206       |
-| 0.6872        | 0.02  | 40   | 0.6790          | 0.0245         | -0.0033          | 0.6875             | 0.0278          | -157.7068      | -203.1401    | -2.3163         | -2.4207       |
-| 0.7024        | 0.03  | 50   | 0.6780          | 0.0314         | -0.0026          | 0.6875             | 0.0340          | -157.6995      | -203.0707    | -2.3164         | -2.4209       |
-
-
 ### Framework versions
 
-- PEFT 0.9.0
+- PEFT 0.10.0
 - Transformers 4.38.2
 - Pytorch 2.0.1+cu117
 - Datasets 2.18.0

adapter_config.json

@@ -6,6 +6,7 @@
   "fan_in_fan_out": false,
   "inference_mode": true,
   "init_lora_weights": true,
+  "layer_replication": null,
   "layers_pattern": null,
   "layers_to_transform": null,
   "loftq_config": {},
@@ -19,8 +20,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "v_pro",
-    "q_proj"
+    "q_proj",
+    "v_pro"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4ad0cb0f4f1f3e22968d5d13c8f4d1c45dac4a914001808254f3a18f470f3f2e
+oid sha256:35bc48fab2cd5d920c7587e239547a271ddb31b7f0ccb6fb3501fa4877cd10a9
 size 8397056

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b1c1f13624741a652e609aa6cdd325797b82ed3542bb4bf68f9052946972941c
+oid sha256:5a71b5468f9b842ef210d99591cc3816d03a929999ec156d8fdb0d3aa41bea7d
 size 4475