feat: 4-bit quantized model

README.md

@@ -1,3 +1,9 @@
 ---
+base_model: ibm/merlinite-7b
+quantized_by: IBM Research
 license: apache-2.0
 ---
+
+# Merlinite 7b - GGUF
+
+4-bit quantized version of [ibm/merlinite-7b](https://huggingface.co/ibm/merlinite-7b)

config.json

@@ -0,0 +1,3 @@
+{
+    "model_type": "mistral"
+}

merlinite-7b-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94f3a16321c9604ca22e970f3b89931ae5b4bbfd4c5d996e2bb606c506590666
+size 4368484576