llama_model_load: error loading model: vocab size mismatch
#8
by
luccazen
- opened
Hi! thanks for the gguf.
i am getting llama_model_load: error loading model: vocab size mismatch
llama_load_model_from_file: failed to load model
when loading any of the quants.
python3 -m llama_cpp.server --model models/Mixtral-8x22B-Instruct-v0.1.Q3_K_M-00001-of-00005.gguf --n_gpu_layers 100 --n_ctx 8192 --host 0.0.0.0
hi,
I am fixing them now
I fixed the issue, kinda mistralai's fault. I am re-uploading them
nice, thanks