Initial release

.gitattributes

@@ -4,6 +4,7 @@
 *.bz2 filter=lfs diff=lfs merge=lfs -text
 *.ckpt filter=lfs diff=lfs merge=lfs -text
 *.ftz filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text
 *.gz filter=lfs diff=lfs merge=lfs -text
 *.h5 filter=lfs diff=lfs merge=lfs -text
 *.joblib filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,52 @@
----
-license: cc-by-nc-4.0
----
+---
+base_model:
+- grimjim/Mistral-7B-Instruct-demi-merge-v0.2-7B
+- grimjim/kunoichi-lemon-royale-7B
+library_name: transformers
+tags:
+- mergekit
+- merge
+license: cc-by-nc-4.0
+pipeline_tag: text-generation
+---
+# kunoichi-lemon-royale-v2-32K-7B-GGUF
+
+This repository contains GGUF quantizations of kunoichi-lemon-royale-v2-32K-7B.
+
+This merge amounts to the grafting of a model derived from Mistral v0.1 (4K sliding window context, to a maximum of 8K practical context length) onto a model derived from Mistral v0.2 (32K context length). It appears to work, although rope_theta in config.json was lowered from 1000000.0 to 100000.0, which works well enough to 16K.
+
+In light testing, this model appears to follow formatting very well, with temperature 1.0 and minP 0.01, using ChatML prompts, even though the underlying model claims to follow Alpaca prompts.
+
+This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+
+- Full weights: [kunoichi-lemon-royale-v2-32K-7B](https://huggingface.co/grimjim/kunoichi-lemon-royale-v2-32K-7B)
+- GGUFs: [kunoichi-lemon-royale-v2-32K-7B-GGUF](https://huggingface.co/grimjim/kunoichi-lemon-royale-v2-32K-7B-GGUF)
+
+## Merge Details
+### Merge Method
+
+This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using [grimjim/Mistral-7B-Instruct-demi-merge-v0.2-7B](https://huggingface.co/grimjim/Mistral-7B-Instruct-demi-merge-v0.2-7B) as a base.
+
+### Models Merged
+
+The following models were included in the merge:
+* [grimjim/kunoichi-lemon-royale-7B](https://huggingface.co/grimjim/kunoichi-lemon-royale-7B)
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+base_model: grimjim/Mistral-7B-Instruct-demi-merge-v0.2-7B
+dtype: bfloat16
+merge_method: task_arithmetic
+slices:
+- sources:
+  - layer_range: [0, 32]
+    model: grimjim/Mistral-7B-Instruct-demi-merge-v0.2-7B
+  - layer_range: [0, 32]
+    model: grimjim/kunoichi-lemon-royale-7B
+    parameters:
+      weight: 1.00
+
+```

kunoichi-lemon-royale-v2-32K-7B.Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8457b5e629f0f4096bb80ddbab23f893ad00f7e46e132ffad29fd35a70353fb
+size 4368439552

kunoichi-lemon-royale-v2-32K-7B.Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c42e60e4ba5f9d098b351efcbf884b3f960d9eae3276df910f5bb773cc27bd3
+size 5131409664

kunoichi-lemon-royale-v2-32K-7B.Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:389074a0a90c1e009843a82191a6d804f85a82831563e1c25db7888bc65858e5
+size 5942065408

kunoichi-lemon-royale-v2-32K-7B.Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:181433e3f25ecf34f0ddcbe81ccd11c3325aa249b902d3bb68d721b8ce058d13
+size 7695857888