paulilioaica
commited on
Commit
•
958c7b2
1
Parent(s):
817811c
Update README.md
Browse files
README.md
CHANGED
@@ -13,23 +13,38 @@ base_model:
|
|
13 |
- rhysjones/phi-2-orange
|
14 |
---
|
15 |
|
16 |
-
# PhiMiX-
|
17 |
|
18 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
19 |
* [cognitivecomputations/dolphin-2_6-phi-2](https://huggingface.co/cognitivecomputations/dolphin-2_6-phi-2)
|
20 |
* [rhysjones/phi-2-orange](https://huggingface.co/rhysjones/phi-2-orange)
|
21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
22 |
## 🧩 Configuration
|
23 |
|
24 |
```yaml
|
25 |
base_model: rhysjones/phi-2-orange
|
26 |
-
gate_mode:
|
27 |
dtype: float16
|
28 |
experts:
|
29 |
- source_model: cognitivecomputations/dolphin-2_6-phi-2
|
30 |
-
positive_prompts: ["
|
31 |
- source_model: rhysjones/phi-2-orange
|
32 |
-
positive_prompts: ["
|
33 |
```
|
34 |
|
35 |
## 💻 Usage
|
@@ -47,11 +62,13 @@ tokenizer = AutoTokenizer.from_pretrained(model)
|
|
47 |
pipeline = transformers.pipeline(
|
48 |
"text-generation",
|
49 |
model=model,
|
50 |
-
|
|
|
51 |
)
|
52 |
|
53 |
-
|
54 |
-
|
55 |
-
outputs = pipeline(
|
56 |
print(outputs[0]["generated_text"])
|
|
|
57 |
```
|
|
|
13 |
- rhysjones/phi-2-orange
|
14 |
---
|
15 |
|
16 |
+
# PhiMiX-2x2B
|
17 |
|
18 |
+
|
19 |
+
## Code is work in progress
|
20 |
+
|
21 |
+
<p align="center">
|
22 |
+
<img src="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11201acc-4089-416d-921b-cbd71fbf8ddb_1024x1024.jpeg" width="500" class="center"/>
|
23 |
+
</p>
|
24 |
+
|
25 |
+
|
26 |
+
PhiMiX-2x2B is a Mixure of Experts (MoE) made with the following models using mergekit:
|
27 |
* [cognitivecomputations/dolphin-2_6-phi-2](https://huggingface.co/cognitivecomputations/dolphin-2_6-phi-2)
|
28 |
* [rhysjones/phi-2-orange](https://huggingface.co/rhysjones/phi-2-orange)
|
29 |
|
30 |
+
|
31 |
+
## ©️ Credits
|
32 |
+
* [mlabonne's phixtral](https://huggingface.co/mlabonne/phixtral-4x2_8) for the PhiConfig and inference code.
|
33 |
+
* [mergekit](https://github.com/cg123/mergekit) code which I tweaked (you can find the PhiConfig [here](https://github.com/cg123/mergekit/blob/508348ae34be17ea0a95d0a288a6e34491a2558a/mergekit/architecture.py#L289))
|
34 |
+
by mainly adding the config in the `moe_mixtral.py` script from `mixtral` branch.
|
35 |
+
|
36 |
+
|
37 |
## 🧩 Configuration
|
38 |
|
39 |
```yaml
|
40 |
base_model: rhysjones/phi-2-orange
|
41 |
+
gate_mode: random
|
42 |
dtype: float16
|
43 |
experts:
|
44 |
- source_model: cognitivecomputations/dolphin-2_6-phi-2
|
45 |
+
positive_prompts: [""]
|
46 |
- source_model: rhysjones/phi-2-orange
|
47 |
+
positive_prompts: [""]
|
48 |
```
|
49 |
|
50 |
## 💻 Usage
|
|
|
62 |
pipeline = transformers.pipeline(
|
63 |
"text-generation",
|
64 |
model=model,
|
65 |
+
trust_remote_code=True,
|
66 |
+
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True,},
|
67 |
)
|
68 |
|
69 |
+
prompt="How many continents are there?"
|
70 |
+
input = f"Instruct: <prompt>\nOutput:"
|
71 |
+
outputs = pipeline(input, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
72 |
print(outputs[0]["generated_text"])
|
73 |
+
|
74 |
```
|