--- library_name: transformers tags: - mergekit - merge - llama base_model: - v000000/L3.1-8B-RP-Test-003-Task_Arithmetic - v000000/L3.1-Niitorm-8B-t0.0001 - Sao10K/L3.1-8B-Niitama-v1.1 - arcee-ai/Llama-3.1-SuperNova-Lite - akjindal53244/Llama-3.1-Storm-8B - arcee-ai/Llama-Spark - v000000/L3.1-8B-RP-Test-002-Task_Arithmetic - grimjim/Llama-3-Instruct-abliteration-LoRA-8B model-index: - name: L3.1-Storniitova-8B results: - task: type: text-generation name: Text Generation dataset: name: IFEval (0-Shot) type: HuggingFaceH4/ifeval args: num_few_shot: 0 metrics: - type: inst_level_strict_acc and prompt_level_strict_acc value: 78.17 name: strict accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/L3.1-Storniitova-8B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: BBH (3-Shot) type: BBH args: num_few_shot: 3 metrics: - type: acc_norm value: 30.81 name: normalized accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/L3.1-Storniitova-8B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MATH Lvl 5 (4-Shot) type: hendrycks/competition_math args: num_few_shot: 4 metrics: - type: exact_match value: 13.29 name: exact match source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/L3.1-Storniitova-8B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GPQA (0-shot) type: Idavidrein/gpqa args: num_few_shot: 0 metrics: - type: acc_norm value: 5.26 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/L3.1-Storniitova-8B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MuSR (0-shot) type: TAUR-Lab/MuSR args: num_few_shot: 0 metrics: - type: acc_norm value: 9.96 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/L3.1-Storniitova-8B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU-PRO (5-shot) type: TIGER-Lab/MMLU-Pro config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 30.84 name: accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/L3.1-Storniitova-8B name: Open LLM Leaderboard --- # Llama-3.1-Storniitova-8B Storniitova-8B is a RP/Instruct model built on the foundation of Llama-3.1-SuperNova-Lite, which is distilled from the 405B parameter variant of Llama-3.1 By only changing the vector tasks, I attempt to retain the full 405B distillation while learning roleplaying capabilties. # (GGUF) mradermacher quants: * [GGUFs](https://huggingface.co/mradermacher/L3.1-Storniitova-8B-GGUF) * [GGUFs imatrix](https://huggingface.co/mradermacher/L3.1-Storniitova-8B-i1-GGUF) # (GGUF) QuantFactory quants: * [GGUFs](https://huggingface.co/QuantFactory/L3.1-Storniitova-8B-GGUF) ----------------------------------------------------------------------------------------------------------- # merge This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit) and other proprietary tools. ## Merge Details ### Merge Method This model was merged using the SLERP, Task_Arithmetic and NEARSWAP merge method. ### Models Merged The following models were included in the merge: * [v000000/L3.1-Niitorm-8B-t0.0001](https://huggingface.co/v000000/L3.1-Niitorm-8B-t0.0001) * [akjindal53244/Llama-3.1-Storm-8B](https://huggingface.co/akjindal53244/Llama-3.1-Storm-8B) * [arcee-ai/Llama-Spark](https://huggingface.co/arcee-ai/Llama-Spark) * [arcee-ai/Llama-3.1-SuperNova-Lite](https://huggingface.co/arcee-ai/Llama-3.1-SuperNova-Lite) * [v000000/L3.1-8B-RP-Test-003-Task_Arithmetic](https://huggingface.co/v000000/L3.1-8B-RP-Test-003-Task_Arithmetic) * [Sao10K/L3.1-8B-Niitama-v1.1](https://huggingface.co/Sao10K/L3.1-8B-Niitama-v1.1) + [grimjim/Llama-3-Instruct-abliteration-LoRA-8B](https://huggingface.co/grimjim/Llama-3-Instruct-abliteration-LoRA-8B) * [v000000/L3.1-8B-RP-Test-002-Task_Arithmetic](https://huggingface.co/v000000/L3.1-8B-RP-Test-002-Task_Arithmetic) + [grimjim/Llama-3-Instruct-abliteration-LoRA-8B](https://huggingface.co/grimjim/Llama-3-Instruct-abliteration-LoRA-8B) ### Recipe The following YAML configuration was used to produce this model: ```yaml #Step1 - Add smarts to Niitama with alchemonaut's algorithm. slices: - sources: - model: Sao10K/L3.1-8B-Niitama-v1.1+grimjim/Llama-3-Instruct-abliteration-LoRA-8B layer_range: [0, 32] - model: akjindal53244/Llama-3.1-Storm-8B layer_range: [0, 32] merge_method: nearswap base_model: Sao10K/L3.1-8B-Niitama-v1.1+grimjim/Llama-3-Instruct-abliteration-LoRA-8B parameters: t: - value: 0.0001 dtype: bfloat16 out_type: float16 #Step 2 - Learn vectors onto Supernova 0.4(Niitorm) models: - model: arcee-ai/Llama-3.1-SuperNova-Lite parameters: weight: 1.0 - model: v000000/L3.1-Niitorm-8B-t0.0001 parameters: weight: 0.4 merge_method: task_arithmetic base_model: arcee-ai/Llama-3.1-SuperNova-Lite parameters: normalize: false dtype: float16 #Step 3 - Fully learn vectors onto Supernova 1.25(Niitorm) models: - model: arcee-ai/Llama-3.1-SuperNova-Lite parameters: weight: 0.0 - model: v000000/L3.1-Niitorm-8B-t0.0001 parameters: weight: 1.25 merge_method: task_arithmetic base_model: arcee-ai/Llama-3.1-SuperNova-Lite parameters: normalize: false dtype: float16 #Step 4 - Merge checkpoints and keep output/input Supernova heavy #Merge with a triangular slerp from sophosympatheia. models: - model: v000000/L3.1-8B-RP-Test-003-Task_Arithmetic merge_method: slerp base_model: v000000/L3.1-8B-RP-Test-002-Task_Arithmetic+grimjim/Llama-3-Instruct-abliteration-LoRA-8B # This model needed some abliteration^ parameters: t: - value: [0, 0, 0.3, 0.4, 0.5, 0.6, 0.5, 0.4, 0.3, 0, 0] dtype: float16 ``` *SLERP distribution* used to smoothly blend the mostly Supernova base with the roleplay vectors: ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64f74b6e6389380c77562762/GP2LMRvMkhVJwNDSEC4oU.png) # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_v000000__L3.1-Storniitova-8B) | Metric |Value| |-------------------|----:| |Avg. |28.06| |IFEval (0-Shot) |78.17| |BBH (3-Shot) |30.81| |MATH Lvl 5 (4-Shot)|13.29| |GPQA (0-shot) | 5.26| |MuSR (0-shot) | 9.96| |MMLU-PRO (5-shot) |30.84|