Create README.md

README.md

@@ -0,0 +1,8 @@
+Quantized version of this: https://huggingface.co/ausboss/llama-30b-supercot
+
+GPTQ quantization using https://github.com/0cc4m/GPTQ-for-LLaMa for compatibility with 0cc4m's fork of KoboldAI
+
+This one is without groupsize to save on VRAM, so that you can enjoy the full 2048 max context if you have 24GB VRAM (or at least get a lot closer to it versus the groupsize version)
+
+Command used to quantize:  
+```python llama.py c:\llama-30b-supercot c4 --wbits 4 --act-order --true-sequential --save_safetensors 4bit.safetensors```