Hugging Face's logo

EricB
/
Llama-3.2-11B-Vision-Instruct-UQFF

uqff
mistral.rs
Model card Files Community
EricB HF staff commited on
Commit
c6ceb2c
•
1 Parent(s): e2fbe02

Update with HQQ4 and HQQ8

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -23,3 +23,5 @@ Run with [mistral.rs](https://github.com/EricLBuehler/mistral.rs). Documentation
23
  |--|--|--|
24
  |llama-3.2-11b-vision-q4k.uqff|Q4K|`./mistralrs-server -i vision-plain -m meta-llama/Llama-3.2-11B-Vision-Instruct -a vllama --from-uqff EricB/Llama-3.2-11B-Vision-Instruct-UQFF/llama-3.2-11b-vision-q4k.uqff`|
25
  |llama-3.2-11b-vision-q8_0.uqff|Q8_0|`./mistralrs-server -i vision-plain -m meta-llama/Llama-3.2-11B-Vision-Instruct -a vllama --from-uqff EricB/Llama-3.2-11B-Vision-Instruct-UQFF/llama-3.2-11b-vision-q8_0.uqff`|
 
 
 
23
  |--|--|--|
24
  |llama-3.2-11b-vision-q4k.uqff|Q4K|`./mistralrs-server -i vision-plain -m meta-llama/Llama-3.2-11B-Vision-Instruct -a vllama --from-uqff EricB/Llama-3.2-11B-Vision-Instruct-UQFF/llama-3.2-11b-vision-q4k.uqff`|
25
  |llama-3.2-11b-vision-q8_0.uqff|Q8_0|`./mistralrs-server -i vision-plain -m meta-llama/Llama-3.2-11B-Vision-Instruct -a vllama --from-uqff EricB/Llama-3.2-11B-Vision-Instruct-UQFF/llama-3.2-11b-vision-q8_0.uqff`|
26
+ |llama-3.2-11b-vision-hqq4.uqff|HQQ4|`./mistralrs-server -i vision-plain -m meta-llama/Llama-3.2-11B-Vision-Instruct -a vllama --from-uqff EricB/Llama-3.2-11B-Vision-Instruct-UQFF/llama-3.2-11b-vision-hqq4.uqff`|
27
+ |llama-3.2-11b-vision-hqq8.uqff|HQQ8|`./mistralrs-server -i vision-plain -m meta-llama/Llama-3.2-11B-Vision-Instruct -a vllama --from-uqff EricB/Llama-3.2-11B-Vision-Instruct-UQFF/llama-3.2-11b-vision-hqq8.uqff`|