Merge branch 'main' of https://huggingface.co/CyberTimon/chimera-7b-4bit-128g
Browse files
README.md
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# This is the 4bit converted version for use with oobabooga's text-generation-webui.
|
| 2 |
+
|
| 3 |
+
All credits go to this amazing project: https://github.com/FreedomIntelligence/LLMZoo
|
| 4 |
+
|
| 5 |
+
Converted with `python llama.py ./chimera-7b c4 --wbits 4 --true-sequential --groupsize 128 --save chimera7b-4bit-128g.pt`
|
| 6 |
+
|
| 7 |
+
It uses groupsize 128. Doesn't use act-order and got quantized with the oobabooga gpt-q branch so it works there.
|
| 8 |
+
|
| 9 |
+
Anyone need a 13b version?
|