asiansoul commited on
Commit
bf85cc1
1 Parent(s): 659c173

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +64 -1
README.md CHANGED
@@ -31,4 +31,67 @@ Isn't it boring to just do LLM?
31
 
32
  <span style="color:purple;font-weight:bold;"> Soon I will open a very cool Streamlit base application based on the model I merged because i am an application engineer. Please wait until then.</span>
33
 
34
- I haven't tested this merge model in depth yet. I'm going to post it here and test it out ^^
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
 
32
  <span style="color:purple;font-weight:bold;"> Soon I will open a very cool Streamlit base application based on the model I merged because i am an application engineer. Please wait until then.</span>
33
 
34
+ I haven't tested this merge model in depth yet. I'm going to post it here and test it out ^^
35
+
36
+ ## Merge Details
37
+ ### Merge Method
38
+
39
+ This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [NousResearch/Meta-Llama-3-8B](https://huggingface.co/NousResearch/Meta-Llama-3-8B) as a base.
40
+
41
+ ### Models Merged
42
+
43
+ The following models were included in the merge:
44
+ * [NousResearch/Hermes-2-Pro-Llama-3-8B](https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B)
45
+ * [cognitivecomputations/dolphin-2.9-llama3-8b](https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b)
46
+ * [Danielbrdz/Barcenas-Llama3-8b-ORPO](https://huggingface.co/Danielbrdz/Barcenas-Llama3-8b-ORPO)
47
+ * [maum-ai/Llama-3-MAAL-8B-Instruct-v0.1](https://huggingface.co/maum-ai/Llama-3-MAAL-8B-Instruct-v0.1)
48
+ * [asiansoul/Llama-3-Open-Ko-Linear-8B](https://huggingface.co/asiansoul/Llama-3-Open-Ko-Linear-8B)
49
+ * [MaziyarPanahi/Llama-3-8B-Instruct-DPO-v0.3](https://huggingface.co/MaziyarPanahi/Llama-3-8B-Instruct-DPO-v0.3)
50
+
51
+ ### Configuration
52
+
53
+ The following YAML configuration was used to produce this model:
54
+
55
+ ```yaml
56
+ models:
57
+ - model: NousResearch/Meta-Llama-3-8B
58
+ # Base model providing a general foundation without specific parameters
59
+
60
+ - model: maum-ai/Llama-3-MAAL-8B-Instruct-v0.1
61
+ parameters:
62
+ density: 0.60
63
+ weight: 0.4
64
+
65
+ - model: asiansoul/Llama-3-Open-Ko-Linear-8B
66
+ parameters:
67
+ density: 0.55
68
+ weight: 0.25
69
+
70
+ - model: MaziyarPanahi/Llama-3-8B-Instruct-DPO-v0.3
71
+ parameters:
72
+ density: 0.55
73
+ weight: 0.15
74
+
75
+ - model: cognitivecomputations/dolphin-2.9-llama3-8b
76
+ parameters:
77
+ density: 0.55
78
+ weight: 0.05
79
+
80
+ - model: Danielbrdz/Barcenas-Llama3-8b-ORPO
81
+ parameters:
82
+ density: 0.55
83
+ weight: 0.125
84
+
85
+ - model: NousResearch/Hermes-2-Pro-Llama-3-8B
86
+ parameters:
87
+ density: 0.55
88
+ weight: 0.125
89
+
90
+ merge_method: dare_ties
91
+ base_model: NousResearch/Meta-Llama-3-8B
92
+ parameters:
93
+ int8_mask: true
94
+ dtype: bfloat16
95
+
96
+
97
+ ```