tsumeone
/

llama-30b-supercot-4bit-cuda

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

tsumeone commited on Apr 23, 2023

Commit

7204437

•

1 Parent(s): 0aa472a

Update README.md

Files changed (1) hide show

README.md +4 -5

README.md CHANGED Viewed

@@ -7,8 +7,7 @@ This one is without groupsize to save on VRAM, so that you can enjoy the full 20
 Command used to quantize:
 ```python llama.py c:\llama-30b-supercot c4 --wbits 4 --act-order --true-sequential --save_safetensors 4bit.safetensors```
-| Eval      | Score |
-| --------- | ----- |
-| WikiText2 | 4.66  |
-| PTB       | 17.64 |
-| C4        | 6.50  |

 Command used to quantize:
 ```python llama.py c:\llama-30b-supercot c4 --wbits 4 --act-order --true-sequential --save_safetensors 4bit.safetensors```
+Evaluation & Score (Lower is better)
+WikiText2: 4.66
+PTB: 17.64
+C4: 6.50