quantize to 4bit
#2
by
Kotokin
- opened
Hi, do you have the ability to quantize it to 4-bit?
TheBloke and Lonestriker should be releasing the full set of quants for this model soon. I made them aware of it.
EDIT: Artefact2 once again was on top of it! See https://huggingface.co/Artefact2/Midnight-Rose-70B-v2.0.3-GGUF
sophosympatheia
changed discussion status to
closed