Update README.md
Browse files
README.md
CHANGED
@@ -1,40 +1,78 @@
|
|
1 |
-
---
|
2 |
-
base_model:
|
3 |
-
|
4 |
-
|
5 |
-
-
|
6 |
-
-
|
7 |
-
|
8 |
-
|
9 |
-
|
10 |
-
|
11 |
-
|
12 |
-
|
13 |
-
|
14 |
-
|
15 |
-
|
16 |
-
|
17 |
-
|
18 |
-
|
19 |
-
|
20 |
-
|
21 |
-
|
22 |
-
|
23 |
-
|
24 |
-
|
25 |
-
|
26 |
-
|
27 |
-
|
28 |
-
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
-
|
34 |
-
|
35 |
-
|
36 |
-
|
37 |
-
|
38 |
-
|
39 |
-
|
40 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
base_model:
|
3 |
+
- Aratako/Antler-7B-RP-v3
|
4 |
+
- Aratako/Japanese-Starling-ChatV-7B-RP
|
5 |
+
- senseable/WestLake-7B-v2
|
6 |
+
- SanjiWatsuki/Kunoichi-DPO-v2-7B
|
7 |
+
- SanjiWatsuki/Silicon-Maid-7B
|
8 |
+
- SanjiWatsuki/Loyal-Macaroni-Maid-7B
|
9 |
+
library_name: transformers
|
10 |
+
tags:
|
11 |
+
- mergekit
|
12 |
+
- merge
|
13 |
+
- not-for-all-audiences
|
14 |
+
- nsfw
|
15 |
+
language:
|
16 |
+
- ja
|
17 |
+
license: apache-2.0
|
18 |
+
---
|
19 |
+
# AntlerStar-RP
|
20 |
+
[GGUF版はこちら/Click here for the GGUF version](https://huggingface.co/Aratako/AntlerStar-RP-GGUF)
|
21 |
+
|
22 |
+
## 概要
|
23 |
+
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
24 |
+
|
25 |
+
[Aratako/Antler-7B-RP-v3](https://huggingface.co/Aratako/Antler-7B-RP-v3)と[Aratako/Japanese-Starling-ChatV-7B-RP](https://huggingface.co/Aratako/Japanese-Starling-ChatV-7B-RP)の2つのモデルをベースにマージして作成したロールプレイ用モデルです。
|
26 |
+
|
27 |
+
## マージの詳細
|
28 |
+
まず、[Aratako/Antler-7B-RP-v3](https://huggingface.co/Aratako/Antler-7B-RP-v3)と[Aratako/Japanese-Starling-ChatV-7B-RP](https://huggingface.co/Aratako/Japanese-Starling-ChatV-7B-RP)の2モデルに対し、以下4モデルのChat Vectorを0.5倍して加算し、各4種類、計8種類のChat Vector加算モデルを作成しました。
|
29 |
+
|
30 |
+
- [senseable/WestLake-7B-v2](https://huggingface.co/senseable/WestLake-7B-v2)
|
31 |
+
- [SanjiWatsuki/Kunoichi-DPO-v2-7B](https://huggingface.co/SanjiWatsuki/Kunoichi-DPO-v2-7B)
|
32 |
+
- [SanjiWatsuki/Silicon-Maid-7B](https://huggingface.co/SanjiWatsuki/Silicon-Maid-7B)
|
33 |
+
- [SanjiWatsuki/Loyal-Macaroni-Maid-7B](https://huggingface.co/SanjiWatsuki/Loyal-Macaroni-Maid-7B)
|
34 |
+
|
35 |
+
次に、このChat Vector加算によってできた各4モデルと元のモデルを、それぞれModel Stockという手法を用い以下のようなconfigを使ってmergekitでマージし、2つのモデルを作成しました。
|
36 |
+
|
37 |
+
```yaml
|
38 |
+
models:
|
39 |
+
- model: ./Antler-7B-RP-v3
|
40 |
+
- model: ./Antler-7B-RP-v3-WestLake-ChatVector
|
41 |
+
- model: ./Antler-7B-RP-v3-Kunoichi-ChatVector
|
42 |
+
- model: ./Antler-7B-RP-v3-SiliconMaid-ChatVector
|
43 |
+
- model: ./Antler-7B-RP-v3-LoyalMacaroniMaid-ChatVector
|
44 |
+
merge_method: model_stock
|
45 |
+
base_model: ./Antler-7B-RP-v3
|
46 |
+
dtype: bfloat16
|
47 |
+
tokenizer_source: union
|
48 |
+
```
|
49 |
+
|
50 |
+
```yaml
|
51 |
+
models:
|
52 |
+
- model: ./Japanese-Starling-ChatV-7B-RP
|
53 |
+
- model: ./Japanese-Starling-ChatV-7B-RP-WestLake-ChatVector
|
54 |
+
- model: ./Japanese-Starling-ChatV-7B-RP-Kunoichi-ChatVector
|
55 |
+
- model: ./Japanese-Starling-ChatV-7B-RP-SiliconMaid-ChatVector
|
56 |
+
- model: ./Japanese-Starling-ChatV-7B-RP-LoyalMacaroniMaid-ChatVector
|
57 |
+
merge_method: model_stock
|
58 |
+
base_model: ./Japanese-Starling-ChatV-7B-RP
|
59 |
+
dtype: bfloat16
|
60 |
+
tokenizer_source: union
|
61 |
+
```
|
62 |
+
|
63 |
+
最後に、この2つのモデルを[DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708)という手法でmergekitを用いてマージしました。
|
64 |
+
|
65 |
+
```yaml
|
66 |
+
models:
|
67 |
+
- model: ./Antler-7B-RP-v3-Model-Stock
|
68 |
+
# no parameters necessary for base model
|
69 |
+
- model: ./Japanese-Starling-ChatV-7B-RP-Model-Stock # follow user intent
|
70 |
+
parameters:
|
71 |
+
density: 1
|
72 |
+
weight: 0.7
|
73 |
+
merge_method: dare_ties
|
74 |
+
base_model: ./Antler-7B-RP-v3-Model-Stock
|
75 |
+
dtype: bfloat16
|
76 |
+
tokenizer_source: union
|
77 |
+
|
78 |
+
```
|