For inference. CPU is enough for both quantization and inference.
ONEKQ AI
company
AI & ML interests
Benchmark, Code Generation, LLM
Organization Card
Edit this README.md markdown file to author your organization card.
models
5
onekq-ai/starcoder2-3b-instruct-v0.1
Text Generation
•
Updated
•
37
onekq-ai/DeepSeek-Coder-V2-Lite-Base-bnb-4bit
Text Generation
•
Updated
•
39
onekq-ai/starcoder2-3b-bnb-4bit
Text Generation
•
Updated
•
73
onekq-ai/starcoder2-7b-bnb-4bit
Text Generation
•
Updated
•
25
onekq-ai/starcoder2-15b-bnb-4bit
Text Generation
•
Updated
•
50
•
1