Update README.md
Browse files
README.md
CHANGED
|
@@ -21,7 +21,7 @@ This repo contains GGUF format model files for [SpeakLeash](https://speakleash.o
|
|
| 21 |
|
| 22 |
Simple Colab notebook for testing: https://colab.research.google.com/drive/1Al9glPVCuOXbtDsks8cMcuzkuu8YDzpg?usp=sharing
|
| 23 |
|
| 24 |
-
### Available quantization formats
|
| 25 |
* **q4_k_m:** Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K
|
| 26 |
* **q5_k_m:** Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q5_K
|
| 27 |
* **q6_k:** Uses Q8_K for all tensors
|
|
|
|
| 21 |
|
| 22 |
Simple Colab notebook for testing: https://colab.research.google.com/drive/1Al9glPVCuOXbtDsks8cMcuzkuu8YDzpg?usp=sharing
|
| 23 |
|
| 24 |
+
### Available quantization formats:
|
| 25 |
* **q4_k_m:** Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K
|
| 26 |
* **q5_k_m:** Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q5_K
|
| 27 |
* **q6_k:** Uses Q8_K for all tensors
|