trust_remote_code=True?
The repository for c4ai-command-r-v01-4bit contains custom code which must be executed to correctly load the model. You can inspect the repository content at c4ai-command-r-v01-4bit.
You can avoid this prompt in future by passing the argument trust_remote_code=True.
Yes, you need to pass trust_remote_code. This is the same with original model.
Yes, you need to pass trust_remote_code. This is the same with original model.
I found how to pass trust_remote_code to load method.
Thank you.
No way to pass trust_remote_code to mlx server (https://github.com/ml-explore/mlx-examples/blob/485180ae912b0a1b242bcdb8880680b072a60017/llms/mlx_lm/server.py).
Is changing code the only way?
Yes, I believe you need to customize the code.
You can open an issue: https://github.com/ml-explore/mlx-examples/tree/main/llms
Sure 👍🏾