Update README.md
Browse files
README.md
CHANGED
@@ -7,9 +7,9 @@ Exllama quant of [Undi95/FlatDolphinMaid-8x7B](https://huggingface.co/Undi95/Fla
|
|
7 |
|
8 |
You probably want this version. It just fits in 24gb of vram at half context (16384).
|
9 |
|
10 |
-
If you really want the larger context [
|
11 |
|
12 |
-
I did make a [
|
13 |
|
14 |
|
15 |
|
|
|
7 |
|
8 |
You probably want this version. It just fits in 24gb of vram at half context (16384).
|
9 |
|
10 |
+
If you really want the larger context [3bpw](https://huggingface.co/Kooten/FlatDolphinMaid-8x7B-3bpw-exl2) should do it but you are probably better of with the gguf version with higher quants.
|
11 |
|
12 |
+
I did make a [4bpw](https://huggingface.co/Kooten/FlatDolphinMaid-8x7B-4bpw-exl2), it might work in a headless or multigpu setup.
|
13 |
|
14 |
|
15 |
|