Hugging Face's logo

sahandrez
/
pairwise-reward-sft-zephyr-7b-sft-qlora-ultrafeedback

PEFT
Safetensors
trl
reward-trainer
Generated from Trainer
Model card Files Community
sahandrez commited on
Commit
2e9e722
•
1 Parent(s): 3ee98d3

Model save

Browse files
Files changed (2) hide show
  1. README.md +84 -0
  2. adapter_model.safetensors +1 -1
README.md ADDED
@@ -0,0 +1,84 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: mistralai/Mistral-7B-v0.1
3
+ library_name: peft
4
+ license: apache-2.0
5
+ metrics:
6
+ - accuracy
7
+ tags:
8
+ - trl
9
+ - reward-trainer
10
+ - generated_from_trainer
11
+ model-index:
12
+ - name: pairwise-reward-sft-zephyr-7b-sft-qlora-ultrafeedback-ultrafeedback-binarized-20241013-124646
13
+ results: []
14
+ ---
15
+
16
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
17
+ should probably proofread and complete it, then remove this comment. -->
18
+
19
+ # pairwise-reward-sft-zephyr-7b-sft-qlora-ultrafeedback-ultrafeedback-binarized-20241013-124646
20
+
21
+ This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on an unknown dataset.
22
+ It achieves the following results on the evaluation set:
23
+ - Loss: 0.4739
24
+ - Accuracy: 0.7592
25
+
26
+ ## Model description
27
+
28
+ More information needed
29
+
30
+ ## Intended uses & limitations
31
+
32
+ More information needed
33
+
34
+ ## Training and evaluation data
35
+
36
+ More information needed
37
+
38
+ ## Training procedure
39
+
40
+ ### Training hyperparameters
41
+
42
+ The following hyperparameters were used during training:
43
+ - learning_rate: 1.5e-05
44
+ - train_batch_size: 16
45
+ - eval_batch_size: 32
46
+ - seed: 42
47
+ - gradient_accumulation_steps: 2
48
+ - total_train_batch_size: 32
49
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
50
+ - lr_scheduler_type: linear
51
+ - num_epochs: 1.0
52
+
53
+ ### Training results
54
+
55
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
56
+ |:-------------:|:------:|:----:|:---------------:|:--------:|
57
+ | 0.6209 | 0.0526 | 100 | 0.6427 | 0.6784 |
58
+ | 0.6346 | 0.1052 | 200 | 0.5829 | 0.7165 |
59
+ | 0.5945 | 0.1578 | 300 | 0.5333 | 0.7351 |
60
+ | 0.5258 | 0.2104 | 400 | 0.5169 | 0.7461 |
61
+ | 0.4914 | 0.2630 | 500 | 0.5209 | 0.7346 |
62
+ | 0.4995 | 0.3155 | 600 | 0.5056 | 0.7536 |
63
+ | 0.5272 | 0.3681 | 700 | 0.5041 | 0.7541 |
64
+ | 0.4993 | 0.4207 | 800 | 0.4943 | 0.7471 |
65
+ | 0.5317 | 0.4733 | 900 | 0.4970 | 0.7602 |
66
+ | 0.5193 | 0.5259 | 1000 | 0.4850 | 0.7597 |
67
+ | 0.4534 | 0.5785 | 1100 | 0.4931 | 0.7582 |
68
+ | 0.4828 | 0.6311 | 1200 | 0.4808 | 0.7582 |
69
+ | 0.5432 | 0.6837 | 1300 | 0.4836 | 0.7491 |
70
+ | 0.4343 | 0.7363 | 1400 | 0.4797 | 0.7582 |
71
+ | 0.4287 | 0.7889 | 1500 | 0.4794 | 0.7612 |
72
+ | 0.5117 | 0.8414 | 1600 | 0.4799 | 0.7587 |
73
+ | 0.4369 | 0.8940 | 1700 | 0.4770 | 0.7582 |
74
+ | 0.4537 | 0.9466 | 1800 | 0.4750 | 0.7566 |
75
+ | 0.451 | 0.9992 | 1900 | 0.4739 | 0.7592 |
76
+
77
+
78
+ ### Framework versions
79
+
80
+ - PEFT 0.12.0
81
+ - Transformers 4.45.2
82
+ - Pytorch 2.4.0+cu121
83
+ - Datasets 2.20.0
84
+ - Tokenizers 0.20.0
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6bb3c8e444293f68c42eea91b15d1a8d80b32d0a5365f09dad56779d4f3fb244
3
  size 1342255048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e03e7540515ace64237f034bf27039f0606155dbf8d7830e042bbf706586d53d
3
  size 1342255048