Hugging Face's logo

HuggingFaceTB
/
SmolLM2-1.7B-Instruct

Text Generation
Transformers
TensorBoard
ONNX
Safetensors
Transformers.js
English
llama
conversational
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
18

Fix q8 weights (use uint8 for q8; int8 produces poor results)

#18
by Xenova HF staff - opened
base: refs/heads/main
←
from: refs/pr/18
Discussion Files changed
+2
-2
Upload fixed q8 ONNX models (reduce_range=True, per_channel=True)06633a3e
Xenova
 Hugging Face TB Research org
•
edited 1 day ago

Slightly better, but not great. Will play around with other settings

Upload folder using huggingface_hub0919b6ca
Fix q8 weights (use uint8 for q8; int8 produces poor results)4f13109a
Xenova changed pull request title from Upload fixed q8 ONNX models (reduce_range=True, per_channel=True) to Fix q8 weights (use uint8 for q8; int8 produces poor results)
Xenova changed pull request status to merged

Sign up or log in to comment