19Leo97/openhermes-mistral-dpo-gptq
Browse files
README.md
CHANGED
@@ -18,15 +18,15 @@ should probably proofread and complete it, then remove this comment. -->
|
|
18 |
|
19 |
This model is a fine-tuned version of [TheBloke/OpenHermes-2-Mistral-7B-GPTQ](https://huggingface.co/TheBloke/OpenHermes-2-Mistral-7B-GPTQ) on the None dataset.
|
20 |
It achieves the following results on the evaluation set:
|
21 |
-
- Loss:
|
22 |
-
- Rewards/chosen: -
|
23 |
-
- Rewards/rejected: -
|
24 |
-
- Rewards/accuracies: 0.
|
25 |
-
- Rewards/margins: -0.
|
26 |
-
- Logps/rejected: -
|
27 |
-
- Logps/chosen: -
|
28 |
-
- Logits/rejected: -2.
|
29 |
-
- Logits/chosen: -2.
|
30 |
|
31 |
## Model description
|
32 |
|
@@ -59,11 +59,11 @@ The following hyperparameters were used during training:
|
|
59 |
|
60 |
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|
61 |
|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
|
62 |
-
| 0.
|
63 |
-
| 0.
|
64 |
-
| 0.
|
65 |
-
| 0.
|
66 |
-
| 0.
|
67 |
|
68 |
|
69 |
### Framework versions
|
|
|
18 |
|
19 |
This model is a fine-tuned version of [TheBloke/OpenHermes-2-Mistral-7B-GPTQ](https://huggingface.co/TheBloke/OpenHermes-2-Mistral-7B-GPTQ) on the None dataset.
|
20 |
It achieves the following results on the evaluation set:
|
21 |
+
- Loss: 0.6864
|
22 |
+
- Rewards/chosen: -0.0176
|
23 |
+
- Rewards/rejected: -0.0077
|
24 |
+
- Rewards/accuracies: 0.3125
|
25 |
+
- Rewards/margins: -0.0098
|
26 |
+
- Logps/rejected: -168.4619
|
27 |
+
- Logps/chosen: -127.7056
|
28 |
+
- Logits/rejected: -2.4072
|
29 |
+
- Logits/chosen: -2.3077
|
30 |
|
31 |
## Model description
|
32 |
|
|
|
59 |
|
60 |
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|
61 |
|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
|
62 |
+
| 0.6768 | 0.005 | 10 | 0.6965 | 0.0206 | -0.0070 | 0.5625 | 0.0276 | -168.4548 | -127.3242 | -2.4001 | -2.3165 |
|
63 |
+
| 0.7062 | 0.01 | 20 | 0.7131 | 0.0043 | 0.0421 | 0.25 | -0.0378 | -167.9636 | -127.4874 | -2.4078 | -2.3156 |
|
64 |
+
| 0.7526 | 0.015 | 30 | 0.7101 | -0.0048 | 0.0478 | 0.125 | -0.0526 | -167.9063 | -127.5779 | -2.4088 | -2.3121 |
|
65 |
+
| 0.6946 | 0.02 | 40 | 0.7003 | -0.0144 | 0.0180 | 0.1875 | -0.0324 | -168.2044 | -127.6736 | -2.4087 | -2.3099 |
|
66 |
+
| 0.6728 | 0.025 | 50 | 0.6864 | -0.0176 | -0.0077 | 0.3125 | -0.0098 | -168.4619 | -127.7056 | -2.4072 | -2.3077 |
|
67 |
|
68 |
|
69 |
### Framework versions
|
adapter_config.json
CHANGED
@@ -20,8 +20,8 @@
|
|
20 |
"rank_pattern": {},
|
21 |
"revision": null,
|
22 |
"target_modules": [
|
23 |
-
"
|
24 |
-
"
|
25 |
],
|
26 |
"task_type": "CAUSAL_LM",
|
27 |
"use_dora": false,
|
|
|
20 |
"rank_pattern": {},
|
21 |
"revision": null,
|
22 |
"target_modules": [
|
23 |
+
"v_proj",
|
24 |
+
"q_proj"
|
25 |
],
|
26 |
"task_type": "CAUSAL_LM",
|
27 |
"use_dora": false,
|
adapter_model.safetensors
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 13648432
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:8c24e16461a1fb6569c731d9d60bbd17dc63e98b71bb9cdf86f6b23b09861e1f
|
3 |
size 13648432
|
runs/May30_20-03-40_bdd30f5aa120/events.out.tfevents.1717099527.bdd30f5aa120.706.0
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:55516823a6c65b92868384aa76233e09ebbad10f9edcfa8dc0190cd00f5bc711
|
3 |
+
size 13580
|
training_args.bin
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 4667
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:61e5075654834112cd13ca383fda6d7ab683d28680cdb6816851a7098415930d
|
3 |
size 4667
|