--- base_model: - SanjiWatsuki/Sonya-7B - EmbeddedLLM/Mistral-7B-Merge-14-v0.1 - SanjiWatsuki/Kunoichi-7B tags: - mergekit - merge --- # merged This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## Merge Details ### Merge Method This model was merged using the SLERP merge method. ### Models Merged The following models were included in the merge: * [SanjiWatsuki/Sonya-7B](https://huggingface.co/SanjiWatsuki/Sonya-7B) * [EmbeddedLLM/Mistral-7B-Merge-14-v0.1](https://huggingface.co/EmbeddedLLM/Mistral-7B-Merge-14-v0.1) * [SanjiWatsuki/Kunoichi-7B](https://huggingface.co/SanjiWatsuki/Kunoichi-7B) ### Configuration The following YAML configuration was used to produce this model: ```yaml base_model: EmbeddedLLM/Mistral-7B-Merge-14-v0.1 dtype: bfloat16 merge_method: slerp parameters: t: - filter: self_attn value: [0.0, 0.5, 0.3, 0.7, 1.0] - filter: mlp value: [1.0, 0.5, 0.7, 0.3, 0.0] - value: 0.5 slices: - sources: - layer_range: [0, 32] model: SanjiWatsuki/Sonya-7B - layer_range: [0, 32] model: SanjiWatsuki/Kunoichi-7B ```