File size: 558 Bytes
b4d0241 3857f20 7c84d30 aea89f2 3857f20 921b8c2 |
1 2 3 4 5 6 7 8 9 10 11 12 |
# This is the 4bit converted version for use with oobabooga's text-generation-webui.
All credits go to this amazing project: https://github.com/FreedomIntelligence/LLMZoo
This is the chat-instruct version
Converted with `python llama.py ./chimera-7b c4 --wbits 4 --true-sequential --groupsize 128 --save chimera7b-4bit-128g.pt`
It uses groupsize 128. Doesn't use act-order and got quantized with the oobabooga gpt-q branch so it works there.
Anyone need a 13b version? (Edit: Can't do it right now as I only get out of memory errors while quantizing.)
|