Hugging Face's logo

hfl
/
chinese-mixtral-gguf

GGUF
Chinese
English
Mixture of Experts
Inference Endpoints
Model card Files
chinese-mixtral-gguf / README.md
hfl-rc's picture
hfl-rc
Update README.md
730e86f verified
preview code
|
raw
history blame
996 Bytes
metadata
license: apache-2.0
language:
  - zh
  - en

Chinese-Mixtral-GGUF

This repository contains the GGUF-v3 models (llama.cpp compatible) for Chinese-Mixtral (this is not a chat/instruction model).

Performance

Metric: PPL, lower is better

Quant PPL
IQ2_XS 6.9784 +/- 0.07476
Q2_K 5.1846 +/- 0.05533
Q3_K 4.5545 +/- 0.04893
Q4_0 4.4917 +/- 0.04844
Q4_K 4.4488 +/- 0.04813
Q5_0 4.4224 +/- 0.04753
Q5_K 4.4192 +/- 0.04768
Q6_K 4.4092 +/- 0.04758
Q8_0 4.4076 +/- 0.04746
F16 x

Due to the file size limitation, for F16 model, please use cat command to concatenate all parts into a single file. You must concatenate these parts in order.

Others

For Hugging Face version, please see: https://huggingface.co/hfl/chinese-mixtral

Please refer to https://github.com/ymcui/Chinese-Mixtral/ for more details.