---
language:
- en
- de
- fr
- it
- pt
- hi
- es
- th
library_name: exllamav2
pipeline_tag: text-generation
tags:
- facebook
- meta
- pytorch
- llama
- llama-3
- safety
license: llama3.2
base_model:
- meta-llama/Llama-Guard-3-1B
---
# Exl2 quants for [Llama-Guard-3-1B](https://huggingface.co/meta-llama/Llama-Guard-3-1B)

This is a quantized model from the llama guard series for use with [exllamav2](https://github.com/turboderp/exllamav2).\
Please refer to the original model card for instructions on how to use this model.

### Only use this version for testing, BF16 weights recommended for deployment or at least 8bpw.

### BPW:

[4.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/4.0bpw)\
[4.5](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/4.5bpw)\
[5.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/5.0bpw)\
[6.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/6.0bpw)\
[6.5](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/6.5bpw)\
[8.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/8.0bpw)\
[measurement.json](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/blob/main/measurement.json)

# How to download:

### oobabooga's downloader

use something like [download-model.py](https://github.com/oobabooga/text-generation-webui/blob/main/download-model.py) to download with python requests.\
Install requirements:

```shell
pip install requests tqdm
```

Example for downloading 8bpw:

```shell
python download-model.py Anthonyg5005/Llama-Guard-3-1B-exl2:8.0bpw
```

### huggingface-cli

You may also use huggingface-cli\
To install it, install python hf-hub

```shell
pip install huggingface-hub
```

Example for 8bpw:

```shell
huggingface-cli download Anthonyg5005/Llama-Guard-3-1B-exl2 --local-dir Llama-Guard-3-1B-exl2-exl2-8bpw --revision 8.0bpw
```
### Git LFS (not recommended)

I would recommend the http downloaders over using git, they can resume downloads if failed and are much easier to work with.\
Make sure to have git and git LFS installed.\
Example for 8bpw download with git:

Have LFS file skip disabled
```shell
# windows
set GIT_LFS_SKIP_SMUDGE=0
# linux
export GIT_LFS_SKIP_SMUDGE=0
```

Clone repo branch
```shell
git clone https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2 -b 8.0bpw
```