Aratako commited on
Commit
0ae5665
1 Parent(s): 80aa31c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +78 -40
README.md CHANGED
@@ -1,40 +1,78 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # Antler-Straling-dare-ties-2
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using ./Antler-7B-RP-v3-Model-Stock as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * ./Japanese-Starling-ChatV-7B-RP-Model-Stock
22
-
23
- ### Configuration
24
-
25
- The following YAML configuration was used to produce this model:
26
-
27
- ```yaml
28
- models:
29
- - model: ./Antler-7B-RP-v3-Model-Stock
30
- # no parameters necessary for base model
31
- - model: ./Japanese-Starling-ChatV-7B-RP-Model-Stock # follow user intent
32
- parameters:
33
- density: 1
34
- weight: 0.7
35
- merge_method: dare_ties
36
- base_model: ./Antler-7B-RP-v3-Model-Stock
37
- dtype: bfloat16
38
- tokenizer_source: union
39
-
40
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - Aratako/Antler-7B-RP-v3
4
+ - Aratako/Japanese-Starling-ChatV-7B-RP
5
+ - senseable/WestLake-7B-v2
6
+ - SanjiWatsuki/Kunoichi-DPO-v2-7B
7
+ - SanjiWatsuki/Silicon-Maid-7B
8
+ - SanjiWatsuki/Loyal-Macaroni-Maid-7B
9
+ library_name: transformers
10
+ tags:
11
+ - mergekit
12
+ - merge
13
+ - not-for-all-audiences
14
+ - nsfw
15
+ language:
16
+ - ja
17
+ license: apache-2.0
18
+ ---
19
+ # AntlerStar-RP
20
+ [GGUF版はこちら/Click here for the GGUF version](https://huggingface.co/Aratako/AntlerStar-RP-GGUF)
21
+
22
+ ## 概要
23
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
24
+
25
+ [Aratako/Antler-7B-RP-v3](https://huggingface.co/Aratako/Antler-7B-RP-v3)と[Aratako/Japanese-Starling-ChatV-7B-RP](https://huggingface.co/Aratako/Japanese-Starling-ChatV-7B-RP)の2つのモデルをベースにマージして作成したロールプレイ用モデルです。
26
+
27
+ ## マージの詳細
28
+ まず、[Aratako/Antler-7B-RP-v3](https://huggingface.co/Aratako/Antler-7B-RP-v3)と[Aratako/Japanese-Starling-ChatV-7B-RP](https://huggingface.co/Aratako/Japanese-Starling-ChatV-7B-RP)の2モデルに対し、以下4モデルのChat Vectorを0.5倍して加算し、各4種類、計8種類のChat Vector加算モデルを作成しました。
29
+
30
+ - [senseable/WestLake-7B-v2](https://huggingface.co/senseable/WestLake-7B-v2)
31
+ - [SanjiWatsuki/Kunoichi-DPO-v2-7B](https://huggingface.co/SanjiWatsuki/Kunoichi-DPO-v2-7B)
32
+ - [SanjiWatsuki/Silicon-Maid-7B](https://huggingface.co/SanjiWatsuki/Silicon-Maid-7B)
33
+ - [SanjiWatsuki/Loyal-Macaroni-Maid-7B](https://huggingface.co/SanjiWatsuki/Loyal-Macaroni-Maid-7B)
34
+
35
+ 次に、このChat Vector加算によってできた各4モデルと元のモデルを、それぞれModel Stockという手法を用い以下のようなconfigを使ってmergekitでマージし、2つのモデルを作成しました。
36
+
37
+ ```yaml
38
+ models:
39
+ - model: ./Antler-7B-RP-v3
40
+ - model: ./Antler-7B-RP-v3-WestLake-ChatVector
41
+ - model: ./Antler-7B-RP-v3-Kunoichi-ChatVector
42
+ - model: ./Antler-7B-RP-v3-SiliconMaid-ChatVector
43
+ - model: ./Antler-7B-RP-v3-LoyalMacaroniMaid-ChatVector
44
+ merge_method: model_stock
45
+ base_model: ./Antler-7B-RP-v3
46
+ dtype: bfloat16
47
+ tokenizer_source: union
48
+ ```
49
+
50
+ ```yaml
51
+ models:
52
+ - model: ./Japanese-Starling-ChatV-7B-RP
53
+ - model: ./Japanese-Starling-ChatV-7B-RP-WestLake-ChatVector
54
+ - model: ./Japanese-Starling-ChatV-7B-RP-Kunoichi-ChatVector
55
+ - model: ./Japanese-Starling-ChatV-7B-RP-SiliconMaid-ChatVector
56
+ - model: ./Japanese-Starling-ChatV-7B-RP-LoyalMacaroniMaid-ChatVector
57
+ merge_method: model_stock
58
+ base_model: ./Japanese-Starling-ChatV-7B-RP
59
+ dtype: bfloat16
60
+ tokenizer_source: union
61
+ ```
62
+
63
+ 最後に、この2つのモデルを[DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708)という手法でmergekitを用いてマージしました。
64
+
65
+ ```yaml
66
+ models:
67
+ - model: ./Antler-7B-RP-v3-Model-Stock
68
+ # no parameters necessary for base model
69
+ - model: ./Japanese-Starling-ChatV-7B-RP-Model-Stock # follow user intent
70
+ parameters:
71
+ density: 1
72
+ weight: 0.7
73
+ merge_method: dare_ties
74
+ base_model: ./Antler-7B-RP-v3-Model-Stock
75
+ dtype: bfloat16
76
+ tokenizer_source: union
77
+
78
+ ```