| # Mixtral 7b 8 Expert | |
|  | |
| This is a preliminary HuggingFace implementation of the newly released MoE model by MistralAi. Make sure to load with `trust_remote_code=True`. | |
| Thanks to @dzhulgakov for his early implementation (https://github.com/dzhulgakov/llama-mistral) that helped me find a working setup. | |
| Come chat about this in our [Disco(rd)](https://discord.gg/S8W8B5nz3v)! :) |