Hugging Face's logo

tsavage68
/
MPT_1000_STEPS_1e7_rate_05_beta_DPO

Text Generation
Transformers
Safetensors
mpt
trl
dpo
Generated from Trainer
custom_code
text-generation-inference
Inference Endpoints
Model card Files Community
tsavage68 commited on
Commit
c28a1e0
•
1 Parent(s): eb92861

End of training

Browse files
Files changed (8) hide show
  1. README.md +28 -28
  2. final_checkpoint/model-00001-of-00003.safetensors +1 -1
  3. final_checkpoint/model-00002-of-00003.safetensors +1 -1
  4. final_checkpoint/model-00003-of-00003.safetensors +1 -1
  5. model-00001-of-00003.safetensors +1 -1
  6. model-00002-of-00003.safetensors +1 -1
  7. model-00003-of-00003.safetensors +1 -1
  8. training_args.bin +1 -1
README.md CHANGED
@@ -17,15 +17,15 @@ should probably proofread and complete it, then remove this comment. -->
17
 
18
  This model is a fine-tuned version of [mosaicml/mpt-7b-instruct](https://huggingface.co/mosaicml/mpt-7b-instruct) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
- - Loss: 0.6924
21
- - Rewards/chosen: -0.0146
22
- - Rewards/rejected: -0.0175
23
  - Rewards/accuracies: 0.5275
24
- - Rewards/margins: 0.0029
25
- - Logps/rejected: -21.6159
26
- - Logps/chosen: -20.8410
27
- - Logits/rejected: 14.2241
28
- - Logits/chosen: 14.2267
29
 
30
  ## Model description
31
 
@@ -59,26 +59,26 @@ The following hyperparameters were used during training:
59
 
60
  | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
61
  |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
62
- | 0.6908 | 0.05 | 50 | 0.6958 | -0.0024 | 0.0016 | 0.4835 | -0.0040 | -21.5521 | -20.8002 | 14.2618 | 14.2644 |
63
- | 0.7007 | 0.1 | 100 | 0.6940 | -0.0004 | -0.0001 | 0.5033 | -0.0003 | -21.5577 | -20.7936 | 14.2508 | 14.2534 |
64
- | 0.6945 | 0.15 | 150 | 0.6935 | -0.0010 | -0.0016 | 0.4923 | 0.0006 | -21.5629 | -20.7956 | 14.2501 | 14.2527 |
65
- | 0.6911 | 0.2 | 200 | 0.6947 | 0.0111 | 0.0130 | 0.5055 | -0.0019 | -21.5142 | -20.7552 | 14.2536 | 14.2561 |
66
- | 0.6944 | 0.24 | 250 | 0.6926 | -0.0007 | -0.0032 | 0.5297 | 0.0025 | -21.5681 | -20.7945 | 14.2489 | 14.2515 |
67
- | 0.6893 | 0.29 | 300 | 0.6925 | -0.0029 | -0.0056 | 0.5143 | 0.0027 | -21.5761 | -20.8017 | 14.2454 | 14.2480 |
68
- | 0.6964 | 0.34 | 350 | 0.6933 | -0.0031 | -0.0043 | 0.4901 | 0.0012 | -21.5718 | -20.8026 | 14.2500 | 14.2526 |
69
- | 0.6846 | 0.39 | 400 | 0.6899 | -0.0142 | -0.0220 | 0.5516 | 0.0078 | -21.6306 | -20.8394 | 14.2259 | 14.2284 |
70
- | 0.6823 | 0.44 | 450 | 0.6910 | -0.0143 | -0.0200 | 0.5143 | 0.0056 | -21.6240 | -20.8400 | 14.2294 | 14.2320 |
71
- | 0.6838 | 0.49 | 500 | 0.6908 | -0.0099 | -0.0159 | 0.5297 | 0.0059 | -21.6103 | -20.8253 | 14.2237 | 14.2263 |
72
- | 0.678 | 0.54 | 550 | 0.6897 | -0.0151 | -0.0234 | 0.5407 | 0.0082 | -21.6354 | -20.8427 | 14.2251 | 14.2277 |
73
- | 0.6872 | 0.59 | 600 | 0.6915 | -0.0176 | -0.0223 | 0.5385 | 0.0047 | -21.6318 | -20.8508 | 14.2284 | 14.2311 |
74
- | 0.6881 | 0.64 | 650 | 0.6906 | -0.0132 | -0.0196 | 0.5319 | 0.0064 | -21.6228 | -20.8362 | 14.2236 | 14.2262 |
75
- | 0.6841 | 0.68 | 700 | 0.6910 | -0.0146 | -0.0202 | 0.5143 | 0.0057 | -21.6249 | -20.8408 | 14.2152 | 14.2178 |
76
- | 0.6883 | 0.73 | 750 | 0.6901 | -0.0148 | -0.0223 | 0.5626 | 0.0075 | -21.6317 | -20.8414 | 14.2218 | 14.2244 |
77
- | 0.6813 | 0.78 | 800 | 0.6917 | -0.0150 | -0.0192 | 0.5341 | 0.0041 | -21.6213 | -20.8422 | 14.2255 | 14.2281 |
78
- | 0.6987 | 0.83 | 850 | 0.6902 | -0.0129 | -0.0204 | 0.5297 | 0.0075 | -21.6253 | -20.8350 | 14.2198 | 14.2223 |
79
- | 0.687 | 0.88 | 900 | 0.6928 | -0.0126 | -0.0148 | 0.5121 | 0.0021 | -21.6067 | -20.8343 | 14.2248 | 14.2275 |
80
- | 0.6885 | 0.93 | 950 | 0.6924 | -0.0146 | -0.0175 | 0.5275 | 0.0029 | -21.6159 | -20.8410 | 14.2241 | 14.2267 |
81
- | 0.6904 | 0.98 | 1000 | 0.6924 | -0.0146 | -0.0175 | 0.5275 | 0.0029 | -21.6159 | -20.8410 | 14.2241 | 14.2267 |
82
 
83
 
84
  ### Framework versions
 
17
 
18
  This model is a fine-tuned version of [mosaicml/mpt-7b-instruct](https://huggingface.co/mosaicml/mpt-7b-instruct) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 0.6919
21
+ - Rewards/chosen: -0.0230
22
+ - Rewards/rejected: -0.0291
23
  - Rewards/accuracies: 0.5275
24
+ - Rewards/margins: 0.0061
25
+ - Logps/rejected: -21.6156
26
+ - Logps/chosen: -20.8382
27
+ - Logits/rejected: 14.2213
28
+ - Logits/chosen: 14.2239
29
 
30
  ## Model description
31
 
 
59
 
60
  | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
61
  |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
62
+ | 0.6958 | 0.05 | 50 | 0.6969 | -0.0103 | -0.0064 | 0.4791 | -0.0040 | -21.5702 | -20.8128 | 14.2683 | 14.2709 |
63
+ | 0.6948 | 0.1 | 100 | 0.6966 | -0.0023 | 0.0014 | 0.5077 | -0.0037 | -21.5546 | -20.7968 | 14.2571 | 14.2597 |
64
+ | 0.6971 | 0.15 | 150 | 0.7007 | -0.0051 | 0.0067 | 0.4681 | -0.0117 | -21.5441 | -20.8024 | 14.2475 | 14.2501 |
65
+ | 0.6891 | 0.2 | 200 | 0.6943 | 0.0187 | 0.0174 | 0.4923 | 0.0013 | -21.5227 | -20.7548 | 14.2452 | 14.2478 |
66
+ | 0.6906 | 0.24 | 250 | 0.6922 | 0.0036 | -0.0018 | 0.4747 | 0.0054 | -21.5609 | -20.7850 | 14.2395 | 14.2421 |
67
+ | 0.6865 | 0.29 | 300 | 0.6942 | 0.0038 | 0.0023 | 0.4857 | 0.0015 | -21.5528 | -20.7845 | 14.2393 | 14.2419 |
68
+ | 0.7058 | 0.34 | 350 | 0.6939 | -0.0025 | -0.0045 | 0.5055 | 0.0020 | -21.5664 | -20.7971 | 14.2533 | 14.2559 |
69
+ | 0.6817 | 0.39 | 400 | 0.6918 | -0.0255 | -0.0318 | 0.5143 | 0.0063 | -21.6210 | -20.8431 | 14.2343 | 14.2369 |
70
+ | 0.6726 | 0.44 | 450 | 0.6902 | -0.0203 | -0.0301 | 0.5582 | 0.0099 | -21.6177 | -20.8327 | 14.2287 | 14.2313 |
71
+ | 0.6927 | 0.49 | 500 | 0.6903 | -0.0159 | -0.0254 | 0.5209 | 0.0096 | -21.6083 | -20.8239 | 14.2329 | 14.2355 |
72
+ | 0.6728 | 0.54 | 550 | 0.6905 | -0.0252 | -0.0342 | 0.5297 | 0.0089 | -21.6258 | -20.8426 | 14.2305 | 14.2331 |
73
+ | 0.6733 | 0.59 | 600 | 0.6877 | -0.0158 | -0.0305 | 0.5341 | 0.0147 | -21.6184 | -20.8237 | 14.2330 | 14.2356 |
74
+ | 0.6937 | 0.64 | 650 | 0.6916 | -0.0222 | -0.0293 | 0.5341 | 0.0071 | -21.6161 | -20.8365 | 14.2242 | 14.2268 |
75
+ | 0.6771 | 0.68 | 700 | 0.6921 | -0.0234 | -0.0294 | 0.5231 | 0.0060 | -21.6163 | -20.8391 | 14.2289 | 14.2315 |
76
+ | 0.6874 | 0.73 | 750 | 0.6916 | -0.0219 | -0.0286 | 0.5121 | 0.0067 | -21.6147 | -20.8361 | 14.2292 | 14.2317 |
77
+ | 0.6772 | 0.78 | 800 | 0.6888 | -0.0187 | -0.0313 | 0.5473 | 0.0127 | -21.6201 | -20.8295 | 14.2308 | 14.2334 |
78
+ | 0.7033 | 0.83 | 850 | 0.6886 | -0.0163 | -0.0294 | 0.5297 | 0.0131 | -21.6163 | -20.8248 | 14.2220 | 14.2245 |
79
+ | 0.6772 | 0.88 | 900 | 0.6894 | -0.0217 | -0.0330 | 0.5297 | 0.0113 | -21.6235 | -20.8357 | 14.2227 | 14.2253 |
80
+ | 0.696 | 0.93 | 950 | 0.6918 | -0.0229 | -0.0293 | 0.5275 | 0.0064 | -21.6160 | -20.8380 | 14.2213 | 14.2239 |
81
+ | 0.6881 | 0.98 | 1000 | 0.6919 | -0.0230 | -0.0291 | 0.5275 | 0.0061 | -21.6156 | -20.8382 | 14.2213 | 14.2239 |
82
 
83
 
84
  ### Framework versions
final_checkpoint/model-00001-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9ddbc2416af7c555f5b6569d06296d51fded87e6201d52416b93d6ec49565243
3
  size 4976746424
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ca80f5fb962748fd20fbbb9a7954f383eb37404b2c21f83bb4382c693551b6d
3
  size 4976746424
final_checkpoint/model-00002-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:501f4a75d860cd8b4972acc1c9175a2901cf54dfd4f3f48ec92a8ee9e01ab46b
3
  size 4966260992
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:737611711e9032ae6ac2965c878faee93eef758c51dccc089a7d39da49e324c6
3
  size 4966260992
final_checkpoint/model-00003-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c0b40fa34bf684412c1165c08848d30a9038a89eab9389392959b9ee51d0cac1
3
  size 3355588232
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f322441b5a22f6598992df039ac1b19d83ac3f697878d9f8c64e4c2beec33f66
3
  size 3355588232
model-00001-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9ddbc2416af7c555f5b6569d06296d51fded87e6201d52416b93d6ec49565243
3
  size 4976746424
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ca80f5fb962748fd20fbbb9a7954f383eb37404b2c21f83bb4382c693551b6d
3
  size 4976746424
model-00002-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:501f4a75d860cd8b4972acc1c9175a2901cf54dfd4f3f48ec92a8ee9e01ab46b
3
  size 4966260992
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:737611711e9032ae6ac2965c878faee93eef758c51dccc089a7d39da49e324c6
3
  size 4966260992
model-00003-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c0b40fa34bf684412c1165c08848d30a9038a89eab9389392959b9ee51d0cac1
3
  size 3355588232
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f322441b5a22f6598992df039ac1b19d83ac3f697878d9f8c64e4c2beec33f66
3
  size 3355588232
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:25e87d6c27f9ec92584326704b8ae805ad3034824688e7ab616a56cc5d6dadd2
3
  size 4475
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d451498134de4439f7d057cbd7d1952c4b4919ca5bb4176ea731caed9fa374e3
3
  size 4475