royleibov commited on
Commit
ab5564a
1 Parent(s): e640cef

Make README conform with ZipNN

Browse files
Files changed (1) hide show
  1. README.md +17 -0
README.md CHANGED
@@ -78,6 +78,23 @@ Importantly, we use a set of hyper-parameters for training that are very differe
78
  - **Base model:** [ibm/granite-7b-base](https://huggingface.co/ibm/granite-7b-base)
79
  - **Teacher Model:** [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)
80
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
81
  ## Prompt Template
82
 
83
  ```python
 
78
  - **Base model:** [ibm/granite-7b-base](https://huggingface.co/ibm/granite-7b-base)
79
  - **Teacher Model:** [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)
80
 
81
+ ## Usage
82
+ This fork is compressed using ZipNN. To use the model, decompress the model tensors as discribed below and load the local weights.
83
+
84
+ You need to [clone this repository](https://huggingface.co/royleibov/Jamba-v0.1-ZipNN-Compressed?clone=true) to decompress the model.
85
+
86
+ Then:
87
+ ```bash
88
+ cd granite-7b-instruct-ZipNN-Compressed
89
+ ```
90
+
91
+ First decompress the model weights:
92
+ ```bash
93
+ python3 zipnn_decompress_path.py --path .
94
+ ```
95
+
96
+ Now just run the local version of the model.
97
+
98
  ## Prompt Template
99
 
100
  ```python