arhanovich commited on
Commit
d0b639e
1 Parent(s): cbd7eb5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -64
README.md CHANGED
@@ -1,64 +0,0 @@
1
- ---
2
- base_model:
3
- - migtissera/SynthIA-70B-v1.2b
4
- - 152334H/miqu-1-70b-sf
5
- - Xwin-LM/Xwin-LM-70B-V0.1
6
- library_name: transformers
7
- tags:
8
- - mergekit
9
- - merge
10
-
11
- ---
12
- # merge
13
-
14
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
-
16
- ## Merge Details
17
- ### Merge Method
18
-
19
- This model was merged using the [linear](https://arxiv.org/abs/2203.05482) merge method.
20
-
21
- ### Models Merged
22
-
23
- The following models were included in the merge:
24
- * [migtissera/SynthIA-70B-v1.2b](https://huggingface.co/migtissera/SynthIA-70B-v1.2b)
25
- * [152334H/miqu-1-70b-sf](https://huggingface.co/152334H/miqu-1-70b-sf)
26
- * [Xwin-LM/Xwin-LM-70B-V0.1](https://huggingface.co/Xwin-LM/Xwin-LM-70B-V0.1)
27
-
28
- ### Configuration
29
-
30
- The following YAML configuration was used to produce this model:
31
-
32
- ```yaml
33
- merge_method: linear
34
- parameters:
35
- weight: 1.0
36
- slices:
37
- - sources:
38
- - model: 152334H/miqu-1-70b-sf
39
- layer_range: [0, 24]
40
- - model: Xwin-LM/Xwin-LM-70B-V0.1
41
- layer_range: [0, 24]
42
- - sources:
43
- - model: migtissera/SynthIA-70B-v1.2b
44
- layer_range: [10, 34]
45
- - sources:
46
- - model: 152334H/miqu-1-70b-sf
47
- layer_range: [25, 49]
48
- - sources:
49
- - model: 152334H/miqu-1-70b-sf
50
- layer_range: [50, 74]
51
- - model: Xwin-LM/Xwin-LM-70B-V0.1
52
- layer_range: [25, 49]
53
- - model: migtissera/SynthIA-70B-v1.2b
54
- layer_range: [35, 59]
55
- - sources:
56
- - model: 152334H/miqu-1-70b-sf
57
- layer_range: [79, 80]
58
- - model: Xwin-LM/Xwin-LM-70B-V0.1
59
- layer_range: [50, 51]
60
- - model: migtissera/SynthIA-70B-v1.2b
61
- layer_range: [60, 61]
62
- dtype: float16
63
- tokenizer_source: model:152334H/miqu-1-70b-sf
64
- ```