TheBloke

/

Mistral-7B-Instruct-v0.2-GGUF

like 406

Hi,

I've encountered an issue while using the Mixtral-8x7B-Instruct-v0.1-GGUF model. Below are the details of my execution command and the error I received.

Command Used:

Using llamafile

llamafile -m mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf --color -c 32768 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "<s>[INST] {prompt} [/INST]"

Using llama.cpp:

 ~/Tools/llama.cpp/main -m mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf --color -c 32768 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "<s>[INST] {prompt} [/INST]"

Error Received:

Both gave this error message

error: create_tensor: tensor 'blk.0.ffn_gate.weight' not found

This error occurs at the conclusion of the process. I'm uncertain if this is an issue with the llamafile command or a problem inherent to the Mixtral model itself.

Could you provide guidance on how to resolve this issue? Any help would be greatly appreciated.

Thanks,
LW