Update README.md
Browse files
README.md
CHANGED
|
@@ -5,6 +5,7 @@ license: cc-by-nc-4.0
|
|
| 5 |
# FlatDolphinMaid-8x7B 3.5bpw
|
| 6 |
Exllama quant of [Undi95/FlatDolphinMaid-8x7B](https://huggingface.co/Undi95/FlatDolphinMaid-8x7B)
|
| 7 |
|
|
|
|
| 8 |
|
| 9 |
### Promt format:
|
| 10 |
|
|
|
|
| 5 |
# FlatDolphinMaid-8x7B 3.5bpw
|
| 6 |
Exllama quant of [Undi95/FlatDolphinMaid-8x7B](https://huggingface.co/Undi95/FlatDolphinMaid-8x7B)
|
| 7 |
|
| 8 |
+
3.5bwp just barely fits in 24gb vram if i lower context (tried with 12288)
|
| 9 |
|
| 10 |
### Promt format:
|
| 11 |
|