Quantization made by Richard Erkhov. [Github](https://github.com/RichardErkhov) [Discord](https://discord.gg/pvy7H8DZMG) [Request more models](https://github.com/RichardErkhov/quant_request) gemma-2-9B-it-advanced-v2.1 - GGUF - Model creator: https://huggingface.co/jsgreenawalt/ - Original model: https://huggingface.co/jsgreenawalt/gemma-2-9B-it-advanced-v2.1/ | Name | Quant method | Size | | ---- | ---- | ---- | | [gemma-2-9B-it-advanced-v2.1.Q2_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q2_K.gguf) | Q2_K | 3.54GB | | [gemma-2-9B-it-advanced-v2.1.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ3_XS.gguf) | IQ3_XS | 3.86GB | | [gemma-2-9B-it-advanced-v2.1.IQ3_S.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ3_S.gguf) | IQ3_S | 4.04GB | | [gemma-2-9B-it-advanced-v2.1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q3_K_S.gguf) | Q3_K_S | 4.04GB | | [gemma-2-9B-it-advanced-v2.1.IQ3_M.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ3_M.gguf) | IQ3_M | 4.19GB | | [gemma-2-9B-it-advanced-v2.1.Q3_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q3_K.gguf) | Q3_K | 4.43GB | | [gemma-2-9B-it-advanced-v2.1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q3_K_M.gguf) | Q3_K_M | 4.43GB | | [gemma-2-9B-it-advanced-v2.1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q3_K_L.gguf) | Q3_K_L | 4.78GB | | [gemma-2-9B-it-advanced-v2.1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ4_XS.gguf) | IQ4_XS | 4.86GB | | [gemma-2-9B-it-advanced-v2.1.Q4_0.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_0.gguf) | Q4_0 | 5.07GB | | [gemma-2-9B-it-advanced-v2.1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ4_NL.gguf) | IQ4_NL | 5.1GB | | [gemma-2-9B-it-advanced-v2.1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_K_S.gguf) | Q4_K_S | 5.1GB | | [gemma-2-9B-it-advanced-v2.1.Q4_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_K.gguf) | Q4_K | 5.37GB | | [gemma-2-9B-it-advanced-v2.1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_K_M.gguf) | Q4_K_M | 5.37GB | | [gemma-2-9B-it-advanced-v2.1.Q4_1.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_1.gguf) | Q4_1 | 5.55GB | | [gemma-2-9B-it-advanced-v2.1.Q5_0.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_0.gguf) | Q5_0 | 6.04GB | | [gemma-2-9B-it-advanced-v2.1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_K_S.gguf) | Q5_K_S | 6.04GB | | [gemma-2-9B-it-advanced-v2.1.Q5_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_K.gguf) | Q5_K | 6.19GB | | [gemma-2-9B-it-advanced-v2.1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_K_M.gguf) | Q5_K_M | 6.19GB | | [gemma-2-9B-it-advanced-v2.1.Q5_1.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_1.gguf) | Q5_1 | 6.52GB | | [gemma-2-9B-it-advanced-v2.1.Q6_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q6_K.gguf) | Q6_K | 7.07GB | | [gemma-2-9B-it-advanced-v2.1.Q8_0.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q8_0.gguf) | Q8_0 | 9.15GB | Original model description: --- base_model: - wzhouad/gemma-2-9b-it-WPO-HB - UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3 - google/gemma-2-9b-it - princeton-nlp/gemma-2-9b-it-SimPO library_name: transformers tags: - mergekit - merge - merge --- # Gemma Advanced V2.1 This is a merge of the 'smartest' advanced fine-tunes available for Gemma-2-9b-it. It includes WPO, SimPO, and SPPO. The merge was performed via the SOTA 'della' merge method. Merge parameters have been hand-tuned for best results. The Q8_0 quant is highly recommended until better quants come along. ## Notes and observations: * The extreme temperature sensitivity from V1 has been fixed, no longer needs to be run at lower temperatures * Has a somewhat different writing style than any of the parent models * Great instruction following * Tracks plot details well and has good situational understanding * Seems to have a good understanding of psychology, emotions and creative writing * More 'sane' than base gemma-it, SPPO, or SimPO - not as prone to 'Cruella De Vil' or 'Evil Sorceress' like SPPO or SimPO, when portraying characters * Would likely serve as a good base for further merges * I'm looking for a job, if you're hiring. I'm a skilled Python developer who brings strong devops skills along with an ever-growing knowledge of machine learning pipelines and models. Message me if you want to talk about what I can bring to your team. * Overall, this feels like a very useful and successful merge. ## Quantized GGUFs can be found here: * [My quants, Q8_0 tested - jsgreenawalt/gemma-2-9B-it-advanced-v2.1-GGUF](https://huggingface.co/jsgreenawalt/gemma-2-9B-it-advanced-v2.1-GGUF) * [iMatrix - mradermacher/gemma-2-9B-it-advanced-v2.1-i1-GGUF](https://huggingface.co/mradermacher/gemma-2-9B-it-advanced-v2.1-i1-GGUF) * [QuantFactory/gemma-2-9B-it-advanced-v2.1-GGUF](https://huggingface.co/QuantFactory/gemma-2-9B-it-advanced-v2.1-GGUF) * [mradermacher/gemma-2-9B-it-advanced-v2.1-GGUF](https://huggingface.co/mradermacher/gemma-2-9B-it-advanced-v2.1-GGUF) Thanks to everyone who was kind enough to provide quants! I'll link to other quants as they appear. # sample ollama Modelfile ```yaml FROM /path/to/file/gemma-2-9B-it-advanced-v2.1-Q8_0.gguf PARAMETER stop "" PARAMETER stop "" PARAMETER num_ctx 8192 TEMPLATE """user {{ if .System }}{{ .System }} {{ end }}{{ .Prompt }} model {{ .Response }}""" ``` This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## Merge Details ### Merge Method This model was merged using the della merge method using [google/gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it) as a base. ### Models Merged The following models were included in the merge: * [wzhouad/gemma-2-9b-it-WPO-HB](https://huggingface.co/wzhouad/gemma-2-9b-it-WPO-HB) * [princeton-nlp/gemma-2-9b-it-SimPO](https://huggingface.co/princeton-nlp/gemma-2-9b-it-SimPO) * [UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3](https://huggingface.co/UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3) ### Configuration The following YAML configuration was used to produce this model: ```yaml models: - model: google/gemma-2-9b-it - model: wzhouad/gemma-2-9b-it-WPO-HB parameters: density: 0.55 weight: 0.6 - model: princeton-nlp/gemma-2-9b-it-SimPO parameters: density: 0.35 weight: 0.6 - model: UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3 parameters: density: 0.25 weight: 0.4 merge_method: della base_model: google/gemma-2-9b-it parameters: normalize: true int8_mask: true lambda: 1.0 epsilon: 0.1 dtype: float16 ```