Llama 3.1 GPTQ, AWQ, and BNB Quants - a hugging-quants Collection

hugging-quants 's Collections

Gemma2 AWQ Quants

Llama 3.2 3B & 1B GGUF Quants

Llama 3.1 GPTQ, AWQ, and BNB Quants

Llama 3.1 GPTQ, AWQ, and BNB Quants

updated Sep 26

Optimised Quants for high-throughput deployments! Compatible with Transformers, TGI & VLLM 🤗

hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4

Text Generation • Updated Sep 13 • 3.76k • 37
hugging-quants/Meta-Llama-3.1-405B-Instruct-BNB-NF4

Text Generation • Updated Sep 16 • 493 • 5
hugging-quants/Meta-Llama-3.1-405B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 17.6k • 16
hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4

Text Generation • Updated Aug 7 • 212k • 87
unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit

Text Generation • Updated Sep 3 • 18.2k • 22
hugging-quants/Meta-Llama-3.1-70B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 49.4k • 17
hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4

Text Generation • Updated Aug 7 • 393k • 59
hugging-quants/Meta-Llama-3.1-8B-Instruct-BNB-NF4

Text Generation • Updated Aug 8 • 3.19k • 7
hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 65.9k • 19