Hugging Face's logo

EricB
/
Mistral-Small-Instruct-2409-UQFF

mistral
uqff
mistral.rs
Model card Files Community
EricB HF staff commited on
Commit
1403598
•
1 Parent(s): a8866ff

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -22,10 +22,10 @@ Run with [mistral.rs](https://github.com/EricLBuehler/mistral.rs). Documentation
22
 
23
  |Quantization type(s)|Example|
24
  |--|--|
25
- |FP8|`./mistralrs-server -i vision-plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-f8e4m3.uqff`|
26
- |HQQ4|`./mistralrs-server -i vision-plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-hqq4.uqff`|
27
- |HQQ8|`./mistralrs-server -i vision-plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-hqq8.uqff`|
28
- |Q3K|`./mistralrs-server -i vision-plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-q3k.uqff`|
29
- |Q4K|`./mistralrs-server -i vision-plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-q4k.uqff`|
30
- |Q5K|`./mistralrs-server -i vision-plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-q5k.uqff`|
31
- |Q8_0|`./mistralrs-server -i vision-plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-q8_0.uqff`|
 
22
 
23
  |Quantization type(s)|Example|
24
  |--|--|
25
+ |FP8|`./mistralrs-server -i plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-f8e4m3.uqff`|
26
+ |HQQ4|`./mistralrs-server -i plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-hqq4.uqff`|
27
+ |HQQ8|`./mistralrs-server -i plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-hqq8.uqff`|
28
+ |Q3K|`./mistralrs-server -i plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-q3k.uqff`|
29
+ |Q4K|`./mistralrs-server -i plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-q4k.uqff`|
30
+ |Q5K|`./mistralrs-server -i plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-q5k.uqff`|
31
+ |Q8_0|`./mistralrs-server -i plain -m EricB/Mistral-Small-Instruct-2409-UQFF --from-uqff mistral-small-2409-instruct-q8_0.uqff`|