kaitchup/Llama-2-7b-gptq-2bit

Model Card for Model ID

This is Meta's Llama 2 7B quantized in 2-bit using AutoGPTQ from Hugging Face Transformers.

Model Details

Model Description

• Developed by: The Kaitchup
• Model type: Causal (Llama 2)
• Language(s) (NLP): English
• License: Apache 2.0, Llama 2 license agreement

Model Sources

The method and code used to quantize the model are explained here: Quantize and Fine-tune LLMs with GPTQ Using Transformers and TRL

Uses

This model is pre-trained and not fine-tuned. You may fine-tune it with PEFT using adapters. Note that the 2-bit quantization significantly decreases the performance of Llama 2.

Other versions

• kaitchup/Llama-2-7b-gptq-4bit
• kaitchup/Llama-2-7b-gptq-3bit

Model Card Contact

The Kaitchup