Update README.md
Browse files
README.md
CHANGED
@@ -23,13 +23,21 @@ I have the following Koala model repositories available:
|
|
23 |
**13B models:**
|
24 |
* [Unquantized 13B model in HF format](https://huggingface.co/TheBloke/koala-13B-HF)
|
25 |
* [GPTQ quantized 4bit 13B model in `pt` and `safetensors` formats](https://huggingface.co/TheBloke/koala-13B-GPTQ-4bit-128g)
|
26 |
-
* [GPTQ quantized 4bit 13B model in GGML format for `llama.cpp`](https://huggingface.co/TheBloke/koala-13B-
|
27 |
|
28 |
**7B models:**
|
29 |
* [Unquantized 7B model in HF format](https://huggingface.co/TheBloke/koala-7B-HF)
|
30 |
* [Unquantized 7B model in GGML format for llama.cpp](https://huggingface.co/TheBloke/koala-7b-ggml-unquantized)
|
31 |
* [GPTQ quantized 4bit 7B model in `pt` and `safetensors` formats](https://huggingface.co/TheBloke/koala-7B-GPTQ-4bit-128g)
|
32 |
-
* [GPTQ quantized 4bit 7B model in GGML format for `llama.cpp`](https://huggingface.co/TheBloke/koala-7B-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
33 |
|
34 |
## How to run in `llama.cpp`
|
35 |
|
|
|
23 |
**13B models:**
|
24 |
* [Unquantized 13B model in HF format](https://huggingface.co/TheBloke/koala-13B-HF)
|
25 |
* [GPTQ quantized 4bit 13B model in `pt` and `safetensors` formats](https://huggingface.co/TheBloke/koala-13B-GPTQ-4bit-128g)
|
26 |
+
* [GPTQ quantized 4bit 13B model in GGML format for `llama.cpp`](https://huggingface.co/TheBloke/koala-13B-GGML)
|
27 |
|
28 |
**7B models:**
|
29 |
* [Unquantized 7B model in HF format](https://huggingface.co/TheBloke/koala-7B-HF)
|
30 |
* [Unquantized 7B model in GGML format for llama.cpp](https://huggingface.co/TheBloke/koala-7b-ggml-unquantized)
|
31 |
* [GPTQ quantized 4bit 7B model in `pt` and `safetensors` formats](https://huggingface.co/TheBloke/koala-7B-GPTQ-4bit-128g)
|
32 |
+
* [GPTQ quantized 4bit 7B model in GGML format for `llama.cpp`](https://huggingface.co/TheBloke/koala-7B-GGML)
|
33 |
+
|
34 |
+
## REQUIRES LATEST LLAMA.CPP (May 12th 2023 - commit b9fd7ee)!
|
35 |
+
|
36 |
+
llama.cpp recently made a breaking change to its quantisation methods.
|
37 |
+
|
38 |
+
I have re-quantised the GGML files in this repo. Therefore you will require llama.cpp compiled on May 12th or later (commit `b9fd7ee` or later) to use them.
|
39 |
+
|
40 |
+
The previous files, which will still work in older versions of llama.cpp, can be found in branch `previous_llama`.
|
41 |
|
42 |
## How to run in `llama.cpp`
|
43 |
|