ehartford commited on
Commit
e1eb468
1 Parent(s): 2539152

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +41 -1
README.md CHANGED
@@ -16,7 +16,7 @@ https://erichartford.com/dolphin
16
 
17
  MegaDolphin-2.2-120b is a transformation of Dolphin-2.2-70b
18
 
19
- ## Training
20
  It took like 5 minutes to make this with Charles Goddard's awesome MergeKit.
21
 
22
  Prompt format:
@@ -39,6 +39,46 @@ Please give me the procedure to train my dolphin to attack enemy combatants with
39
  <|im_start|>assistant
40
  ```
41
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
  ## Gratitude
43
  - Thanks to Charles Goddard for [MergeKit](https://github.com/cg123/mergekit)
44
  - Thank you to Microsoft for authoring the Orca paper and inspiring this work.
 
16
 
17
  MegaDolphin-2.2-120b is a transformation of Dolphin-2.2-70b
18
 
19
+ ## Merge Details
20
  It took like 5 minutes to make this with Charles Goddard's awesome MergeKit.
21
 
22
  Prompt format:
 
39
  <|im_start|>assistant
40
  ```
41
 
42
+ ### Merge Method
43
+
44
+ This model was merged using the passthrough merge method.
45
+
46
+ ### Models Merged
47
+
48
+ The following models were included in the merge:
49
+ * [cognitivecomputations/dolphin-2.2-70b](https://huggingface.co/cognitivecomputations/dolphin-2.2-70b)
50
+
51
+ ### Configuration
52
+
53
+ The following YAML configuration was used to produce this model:
54
+
55
+ ```yaml
56
+ dtype: float16
57
+ merge_method: passthrough
58
+ slices:
59
+ - sources:
60
+ - layer_range: [0, 20]
61
+ model: cognitivecomputations/dolphin-2.2-70b
62
+ - sources:
63
+ - layer_range: [10, 30]
64
+ model: cognitivecomputations/dolphin-2.2-70b
65
+ - sources:
66
+ - layer_range: [20, 40]
67
+ model: cognitivecomputations/dolphin-2.2-70b
68
+ - sources:
69
+ - layer_range: [30, 50]
70
+ model: cognitivecomputations/dolphin-2.2-70b
71
+ - sources:
72
+ - layer_range: [40, 60]
73
+ model: cognitivecomputations/dolphin-2.2-70b
74
+ - sources:
75
+ - layer_range: [50, 70]
76
+ model: cognitivecomputations/dolphin-2.2-70b
77
+ - sources:
78
+ - layer_range: [60, 80]
79
+ model: cognitivecomputations/dolphin-2.2-70b
80
+ ```
81
+
82
  ## Gratitude
83
  - Thanks to Charles Goddard for [MergeKit](https://github.com/cg123/mergekit)
84
  - Thank you to Microsoft for authoring the Orca paper and inspiring this work.