Make README conform with ZipNN
Browse files
README.md
CHANGED
@@ -78,6 +78,23 @@ Importantly, we use a set of hyper-parameters for training that are very differe
|
|
78 |
- **Base model:** [ibm/granite-7b-base](https://huggingface.co/ibm/granite-7b-base)
|
79 |
- **Teacher Model:** [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)
|
80 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
81 |
## Prompt Template
|
82 |
|
83 |
```python
|
|
|
78 |
- **Base model:** [ibm/granite-7b-base](https://huggingface.co/ibm/granite-7b-base)
|
79 |
- **Teacher Model:** [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)
|
80 |
|
81 |
+
## Usage
|
82 |
+
This fork is compressed using ZipNN. To use the model, decompress the model tensors as discribed below and load the local weights.
|
83 |
+
|
84 |
+
You need to [clone this repository](https://huggingface.co/royleibov/Jamba-v0.1-ZipNN-Compressed?clone=true) to decompress the model.
|
85 |
+
|
86 |
+
Then:
|
87 |
+
```bash
|
88 |
+
cd granite-7b-instruct-ZipNN-Compressed
|
89 |
+
```
|
90 |
+
|
91 |
+
First decompress the model weights:
|
92 |
+
```bash
|
93 |
+
python3 zipnn_decompress_path.py --path .
|
94 |
+
```
|
95 |
+
|
96 |
+
Now just run the local version of the model.
|
97 |
+
|
98 |
## Prompt Template
|
99 |
|
100 |
```python
|