Update README.md
Browse files
README.md
CHANGED
@@ -14,6 +14,15 @@ quantized_by: ilsp
|
|
14 |
|
15 |
![image/png](https://miro.medium.com/v2/resize:fit:720/format:webp/1*IaE7RJk6JffW8og-MOnYCA.png)
|
16 |
|
17 |
-
|
18 |
|
19 |
In this repository you can find quantised GGUF variants of [ILSP's Meltemi-7B-Instruct-v1](https://huggingface.co/ilsp/Meltemi-7B-Instruct-v1) model, created using [llama.cpp](https://github.com/ggerganov/llama.cpp).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
14 |
|
15 |
![image/png](https://miro.medium.com/v2/resize:fit:720/format:webp/1*IaE7RJk6JffW8og-MOnYCA.png)
|
16 |
|
17 |
+
## Description
|
18 |
|
19 |
In this repository you can find quantised GGUF variants of [ILSP's Meltemi-7B-Instruct-v1](https://huggingface.co/ilsp/Meltemi-7B-Instruct-v1) model, created using [llama.cpp](https://github.com/ggerganov/llama.cpp).
|
20 |
+
|
21 |
+
## Provided files (Use case column taken from the llama.cpp documentation)
|
22 |
+
|
23 |
+
Based on the information
|
24 |
+
|
25 |
+
| Name | Quant method | Bits | Size | Appr. RAM required | Use case |
|
26 |
+
| ---- | ---- | ---- | ---- | ---- | ----- |
|
27 |
+
| [meltemi-instruct-v1_q3_K_M.bin](https://huggingface.co/ilsp/Meltemi-7B-Instruct-v1-GGUF/blob/main/meltemi-instruct-v1_q3_K_M.bin) | Q3_K_M | 3 | 3.67 GB| 6.45 GB | very small, very high quality loss |
|
28 |
+
| [meltemi-instruct-v1_q5_K_M.bin](https://huggingface.co/ilsp/Meltemi-7B-Instruct-v1-GGUF/blob/main/meltemi-instruct-v1_q5_K_M.bin) | Q5_K_M | 5 | 5.31 GB| 8.1 GB | large, very low quality loss - recommended |
|