---
license: apache-2.0
tags:
- moe
- mergekit
---

# Beyonder-4x7b

This model is a Mixure of Experts (MoE) made with [mergekit](https://github.com/cg123/mergekit) (mixtral branch). It uses the following base models:
 * [openchat/openchat-3.5-1210](https://huggingface.co/openchat/openchat-3.5-1210)
 * [beowolx/CodeNinja-1.0-OpenChat-7B](https://huggingface.co/beowolx/CodeNinja-1.0-OpenChat-7B)
 * [maywell/PiVoT-0.1-Starling-LM-RP](https://huggingface.co/maywell/PiVoT-0.1-Starling-LM-RP)
 * [WizardLM/WizardMath-7B-V1.1](https://huggingface.co/WizardLM/WizardMath-7B-V1.1)

## 🧩 Configuration

```yaml
base_model: openchat/openchat-3.5-1210
gate_mode: hidden
experts:
  - source_model: openchat/openchat-3.5-1210
    positive_prompts:
    - "chat"
    - "assistant"
    - "tell me"
    - "explain"
    negative_prompts:
    - "storywriting"
    - "mathematics"
    - "reasoning"
    - "code"
    - "programming"
  - source_model: beowolx/CodeNinja-1.0-OpenChat-7B
    positive_prompts:
    - "code"
    - "python"
    - "javascript"
    - "programming"
    - "algorithm"
    negative_prompts:
    - "chat"
    - "assistant"
    - "storywriting"
    - "mathematics"
    - "reasoning"
  - source_model: maywell/PiVoT-0.1-Starling-LM-RP
    positive_prompts:
    - "storywriting"
    - "write"
    - "scene"
    - "story"
    - "character"
    negative_prompts:
    - "chat"
    - "assistant"
    - "code"
    - "programming"
    - "mathematics"
    - "reasoning"
  - source_model: WizardLM/WizardMath-7B-V1.1
    positive_prompts:
    - "reason"
    - "math"
    - "mathematics"
    - "solve"
    - "count"
    negative_prompts:
    - "chat"
    - "assistant"
    - "code"
    - "programming"
    - "storywriting"
```