jeiku commited on
Commit
0a11386
1 Parent(s): 04677f5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -0
README.md CHANGED
@@ -9,6 +9,30 @@ tags:
9
  - merge
10
 
11
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  # rainbow
13
 
14
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
 
9
  - merge
10
 
11
  ---
12
+
13
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/626dfb8786671a29c715f8a9/uAfKDeavEBisJYmDxjJsE.png)
14
+
15
+ technicolor consists of the following merge, which was then merged with the below LoRAs to produce rainbow:
16
+
17
+ ```yaml
18
+ slices:
19
+ - sources:
20
+ - model: paulml/OGNO-7B
21
+ layer_range: [0, 32]
22
+ - model: SanjiWatsuki/Kunoichi-DPO-v2-7B
23
+ layer_range: [0, 32]
24
+ merge_method: slerp
25
+ base_model: SanjiWatsuki/Kunoichi-DPO-v2-7B
26
+ parameters:
27
+ t:
28
+ - filter: self_attn
29
+ value: [0, 0.5, 0.3, 0.7, 1]
30
+ - filter: mlp
31
+ value: [1, 0.5, 0.7, 0.3, 0]
32
+ - value: 0.5
33
+ dtype: bfloat16
34
+ ```
35
+
36
  # rainbow
37
 
38
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).