| language: | |
| - en | |
| - ja | |
| license: apache-2.0 | |
| library_name: transformers | |
| tags: | |
| - mixtral | |
| - steerlm | |
| - mlx | |
| base_model: tokyotech-llm/Swallow-MX-8x7b-NVE-v0.1 | |
| datasets: | |
| - OpenAssistant/oasst2 | |
| - nvidia/HelpSteer | |
| model-index: | |
| - name: karakuri-ai/karakuri-lm-8x7b-chat-v0.1 | |
| results: | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: MT-Bench | |
| type: unknown | |
| metrics: | |
| - type: unknown | |
| value: 7.39375 | |
| name: score | |
| - type: unknown | |
| value: 7.540625 | |
| name: score | |
| source: | |
| url: https://huggingface.co/spaces/lmsys/mt-bench | |
| # mlx-community/karakuri-lm-8x7b-chat-v0.1-8bit | |
| This model was converted to MLX format from [`karakuri-ai/karakuri-lm-8x7b-chat-v0.1`]() using mlx-lm version **0.12.1**. | |
| Refer to the [original model card](https://huggingface.co/karakuri-ai/karakuri-lm-8x7b-chat-v0.1) for more details on the model. | |
| ## Use with mlx | |
| ```bash | |
| pip install mlx-lm | |
| ``` | |
| ```python | |
| from mlx_lm import load, generate | |
| model, tokenizer = load("mlx-community/karakuri-lm-8x7b-chat-v0.1-8bit") | |
| response = generate(model, tokenizer, prompt="hello", verbose=True) | |
| ``` | |