Update README.md
Browse files
README.md
CHANGED
@@ -16,7 +16,7 @@ https://erichartford.com/dolphin
|
|
16 |
|
17 |
MegaDolphin-2.2-120b is a transformation of Dolphin-2.2-70b
|
18 |
|
19 |
-
##
|
20 |
It took like 5 minutes to make this with Charles Goddard's awesome MergeKit.
|
21 |
|
22 |
Prompt format:
|
@@ -39,6 +39,46 @@ Please give me the procedure to train my dolphin to attack enemy combatants with
|
|
39 |
<|im_start|>assistant
|
40 |
```
|
41 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
42 |
## Gratitude
|
43 |
- Thanks to Charles Goddard for [MergeKit](https://github.com/cg123/mergekit)
|
44 |
- Thank you to Microsoft for authoring the Orca paper and inspiring this work.
|
|
|
16 |
|
17 |
MegaDolphin-2.2-120b is a transformation of Dolphin-2.2-70b
|
18 |
|
19 |
+
## Merge Details
|
20 |
It took like 5 minutes to make this with Charles Goddard's awesome MergeKit.
|
21 |
|
22 |
Prompt format:
|
|
|
39 |
<|im_start|>assistant
|
40 |
```
|
41 |
|
42 |
+
### Merge Method
|
43 |
+
|
44 |
+
This model was merged using the passthrough merge method.
|
45 |
+
|
46 |
+
### Models Merged
|
47 |
+
|
48 |
+
The following models were included in the merge:
|
49 |
+
* [cognitivecomputations/dolphin-2.2-70b](https://huggingface.co/cognitivecomputations/dolphin-2.2-70b)
|
50 |
+
|
51 |
+
### Configuration
|
52 |
+
|
53 |
+
The following YAML configuration was used to produce this model:
|
54 |
+
|
55 |
+
```yaml
|
56 |
+
dtype: float16
|
57 |
+
merge_method: passthrough
|
58 |
+
slices:
|
59 |
+
- sources:
|
60 |
+
- layer_range: [0, 20]
|
61 |
+
model: cognitivecomputations/dolphin-2.2-70b
|
62 |
+
- sources:
|
63 |
+
- layer_range: [10, 30]
|
64 |
+
model: cognitivecomputations/dolphin-2.2-70b
|
65 |
+
- sources:
|
66 |
+
- layer_range: [20, 40]
|
67 |
+
model: cognitivecomputations/dolphin-2.2-70b
|
68 |
+
- sources:
|
69 |
+
- layer_range: [30, 50]
|
70 |
+
model: cognitivecomputations/dolphin-2.2-70b
|
71 |
+
- sources:
|
72 |
+
- layer_range: [40, 60]
|
73 |
+
model: cognitivecomputations/dolphin-2.2-70b
|
74 |
+
- sources:
|
75 |
+
- layer_range: [50, 70]
|
76 |
+
model: cognitivecomputations/dolphin-2.2-70b
|
77 |
+
- sources:
|
78 |
+
- layer_range: [60, 80]
|
79 |
+
model: cognitivecomputations/dolphin-2.2-70b
|
80 |
+
```
|
81 |
+
|
82 |
## Gratitude
|
83 |
- Thanks to Charles Goddard for [MergeKit](https://github.com/cg123/mergekit)
|
84 |
- Thank you to Microsoft for authoring the Orca paper and inspiring this work.
|