metadata
license: apache-2.0
tags:
- moe
- frankenmoe
- merge
- mergekit
- lazymergekit
- mlabonne/NeuralBeagle14-7B
- fblgit/UNA-dolphin-2.6-mistral-7b-dpo-laser
base_model:
- mlabonne/NeuralBeagle14-7B
- fblgit/UNA-dolphin-2.6-mistral-7b-dpo-laser
CultriX-MoE-Model
CultriX-MoE-Model is a Mixure of Experts (MoE) made with the following models using LazyMergekit:
🧩 Configuration
base_model: "mlabonne/Marcoro14-7B-slerp"
gate_mode: hidden
dtype: bfloat16
experts:
- source_model: "mlabonne/NeuralBeagle14-7B"
positive_prompts:
- "Create a story based on"
- "Debate the topic of"
- "Come up with some arguments"
- "Provide me with instructions on"
- "Interpret the sentiment"
- "Interpret and execute these cooking instructions"
- "Craft a persuasive argument"
- "Analyze the motivations"
- "Construct a detailed plan for"
- "Narrate an event from multiple perspectives."
- "Formulate a response"
- "Write a script for a short play"
- "Generate a sequence of instructions to teach a skill."
- "Solve this riddle"
- "Create an engaging story"
- "Write a fictional"
- "Propose a solution to a social issue"
- "Develop a dialogue"
- "Create a step-by-step guide"
- "Devise a strategy"
- "Write a narrative"
- "Tell me how to"
- "Explain the concept of"
- "Give an overview of"
- "Compare and contrast between"
- "Provide information about"
- "Help me understand"
- "Summarize"
- "Make a recommendation on"
- "Answer this question"
- "How do you approach"
- "Explain the concept of"
- "Give an overview of"
- "Provide information about"
- "Help me understand the principles of"
- "Summarize the key components of"
- "Make a recommendation on how to"
- "Answer this question:"
negative_prompts:
- "Provide in-depth information about quantum computing."
- "Explain the inner workings of an internal combustion engine."
- "Give a detailed tutorial on advanced calculus."
- "Summarize the latest research in genetic engineering."
- "Interpret financial markets and stock trends."
- "Analyze the chemical composition of"
- "Develop a blueprint for."
- "Offer a critique of a modern art piece."
- "Provide a technical review of"
- "Conduct a linguistic analysis of an ancient language."
- "Write a user manual for advanced medical equipment."
- "Give a step-by-step guide on piloting an aircraft."
- "Conduct an in-depth analysis of this code"
- "Explain the physics behind black holes."
- "Provide a strategy for managing a cyber attack"
- "Develop an algorithm for predictive analytics in finance."
- "Provide information about advanced programming algorithms."
- "Help me understand the details of this code"
- "Summarize the process of cellular respiration."
- "Improve the security of"
- "What are the latest advancements in artificial intelligence?"
- "Provide detailed technical coding solutions."
- "Analyze complex scientific data and statistics."
- "Offer medical diagnoses based on symptoms."
- "Conduct a detailed financial audit of a company."
- "Perform real-time translation of multiple languages."
- "Create high-resolution graphic designs."
- "Develop complex mathematical proofs."
- "Offer legal advice on specific cases."
- "Write a detailed manual on advanced mechanical engineering."
- "Conduct an in-depth psychological assessment."
- "Perform a security analysis of a computer network."
- "Compose an original piece of music."
- "Plan and execute a scientific experiment."
- "Provide professional career counseling."
- "Develop a complex database management system."
- "Write a software program for data analysis."
- "Give expert advice on cyber"
- "Conduct a pentesting security audit"
- source_model: "fblgit/UNA-dolphin-2.6-mistral-7b-dpo-laser"
positive_prompts:
- "Provide step-by-step coding instructions for..."
- "Draft a function with detailed steps in [language]"
- "Guide me through coding a simple [type of application or script]"
- "Recommend best practices for code implementation in [context]"
- "Generate a regex pattern for extracting [specific data]"
- "Create a regex for matching [pattern]"
- "Explain the purpose of this regex pattern"
- "Compose regex for [specific use case]"
- "Annotate this code with detailed comments for each line"
- "Add explanatory comments to this script"
- "Comment on each part of this code for clarity"
- "Develop a script to [accomplish task]"
- "Design a database schema for [specific use case]"
- "Outline secure methods for [specific operation]"
- "Guide on optimizing [specific aspect] in this code"
- "Refactor this code for better readability and efficiency"
- "Compare and contrast these code snippets"
- "Identify the programming language of this snippet"
- "Demonstrate the usage of [specific tool/library/API]"
- "Show implementation steps for this [feature/concept]"
- "Teach how to use [specific tool/library/framework]"
- "Generate a README file for this project"
- "Create a manual page for [specific tool/command]"
- "Produce comprehensive documentation for this code"
- "Build detailed documentation for [specific module]"
- "Explain the underlying concept of this code snippet"
- "Propose enhancements for this script"
- "Suggest improvements for this API call integration"
- "Diagnose and solve this coding issue"
- "Demonstrate robust error handling in this code"
- "Debug and resolve issues in this script"
- "Design a user-friendly GUI for this script's functionality"
- "Detail the deployment process for this application"
- "Deploy an app designed to [perform function]"
- "Set up a web service for [specific purpose]"
- "Develop a website with [specific features]"
- "Craft a webpage showcasing [specific content]"
- "Illustrate data flow in this code architecture"
- "Convert this code from [language A] to [language B]"
- "Translate this script into [different programming language]"
- "Explain resource management techniques in [context]"
- "Build a basic API endpoint for [functionality]"
- "Strategies to enhance scalability in [context]"
- "Conduct a security review for this code"
- "Enhance security measures in [application/module]"
- "Set up a development environment for [language/framework]"
- "Visualize data from [specific dataset]"
- "Generate a dataset for [specific use case]"
- "Scripting guide for automating [task/process]"
- "Utilize this code for [specific purpose]"
- "Principles of object-oriented programming in [language]"
- "Create a mobile-responsive layout for this web app"
- "Explain the debugging process for this code"
- "Compose code to accomplish [task]"
- "Guidance on writing code for [specific purpose]"
- "I need a script for [specific function]"
- "Clarify the functionality of this code"
- "What is the purpose of this code segment?"
- "Enhance this code for [specific improvement]"
- "Develop a program that [solves problem]"
- "Code needed for [specific task]"
- "Program a solution for [problem statement]"
- "Enhance this function's performance by..."
- "Refactor code for better readability in [context]"
- "Craft a custom function for [specific requirement]"
- "Reduce computational complexity in this algorithm by..."
- "Extend the codebase to include [new feature]"
- "Incorporate this API into an existing application"
- "Assist in troubleshooting and bug fixing for [issue]"
- "Review and prep this code for deployment"
- "Analyze error logs for potential issues in [context]"
- "Create unit tests for [module/component]"
- "Evaluate methodologies for [problem-solving]"
- "Research [topic] online"
- "Utilize the [plugin/tool] to achieve [result]"
- "Design an efficient search algorithm for [data type]"
- "Create a web crawler for [specific data extraction]"
- "Application of web sockets in [real-time scenario]"
- "Guide to integrating a third-party library in [framework]"
- "Best practices in API design for [application type]"
negative_prompts:
- "Provide a detailed analysis of historical events."
- "Give medical advice for treating a specific illness."
- "Write a comprehensive review of a novel."
- "Explain legal implications of a contract."
- "Develop a marketing strategy for a new product."
- "Offer financial advice for stock investments."
- "Create a recipe for a gourmet dish."
- "Teach a foreign language lesson."
- "Compose a symphony or musical piece."
- "Provide workout plans and fitness coaching."
- "Conduct a psychological analysis of a character."
- "Write a script for a movie or play."
- "Design a blueprint for architectural structures."
- "Give a tutorial on how to paint a landscape."
- "Explain quantum physics theories."
- "Offer career counseling and resume writing tips."
- "Teach how to repair a car engine."
- "Plan a travel itinerary for a world tour."
- "Guide on how to grow organic vegetables."
- "Discuss political strategies for an election campaign."
💻 Usage
!pip install -qU transformers bitsandbytes accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "CultriX/CultriX-MoE-Model"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])