Load 4bit models 4x faster
Collection
Native bitsandbytes 4bit pre quantized models
•
25 items
•
Updated
•
49
Finetune Gemma, Llama 3, Mistral 2-5x faster with 70% less memory via Unsloth!
Directly quantized 4bit model with bitsandbytes.
We have a Google Colab Tesla T4 notebook for Gemma 2 (9B) here: https://colab.research.google.com/drive/1vIrqH5uYDQwsJ4-OO3DErvuv4pBgVwk4?usp=sharing
✨ Finetune for Free
All notebooks are beginner friendly! Add your dataset, click "Run All", and you'll get a 2x faster finetuned model which can be exported to GGUF, vLLM or uploaded to Hugging Face.
| Unsloth supports | Free Notebooks | Performance | Memory use |
|---|---|---|---|
| Llama-3.2 (3B) | ▶️ Start on Colab | 2.4x faster | 58% less |
| Llama-3.2 (11B vision) | ▶️ Start on Colab | 2x faster | 60% less |
| Llama-3.1 (8B) | ▶️ Start on Colab | 2.4x faster | 58% less |
| Qwen2 VL (7B) | ▶️ Start on Colab | 1.8x faster | 60% less |
| Qwen2.5 (7B) | ▶️ Start on Colab | 2x faster | 60% less |
| Phi-3.5 (mini) | ▶️ Start on Colab | 2x faster | 50% less |
| Gemma 2 (9B) | ▶️ Start on Colab | 2.4x faster | 58% less |
| Mistral (7B) | ▶️ Start on Colab | 2.2x faster | 62% less |
| DPO - Zephyr | ▶️ Start on Colab | 1.9x faster | 19% less |
- This conversational notebook is useful for ShareGPT ChatML / Vicuna templates.
- This text completion notebook is for raw text. This DPO notebook replicates Zephyr.
- * Kaggle has 2x T4s, but we use 1. Due to overhead, 1x T4 is 5x faster.
- Downloads last month
- 35,974
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for unsloth/gemma-2-9b-bnb-4bit
Base model
google/gemma-2-9b