Hugging Face's logo

bartowski
/
Einstein-v5-v0.2-7B-GGUF

Text Generation
GGUF
axolotl
Generated from Trainer
Mistral
instruct
finetune
chatml
gpt4
synthetic data
science
physics
chemistry
biology
math
Inference Endpoints
conversational
Model card Files Community
bartowski commited on
Commit
d63837f
•
1 Parent(s): f64b34c

Llamacpp quants

Browse files
Files changed (18) hide show
  1. .gitattributes +16 -0
  2. Einstein-v5-v0.2-7B-IQ3_M.gguf +3 -0
  3. Einstein-v5-v0.2-7B-IQ3_S.gguf +3 -0
  4. Einstein-v5-v0.2-7B-IQ4_NL.gguf +3 -0
  5. Einstein-v5-v0.2-7B-IQ4_XS.gguf +3 -0
  6. Einstein-v5-v0.2-7B-Q2_K.gguf +3 -0
  7. Einstein-v5-v0.2-7B-Q3_K_L.gguf +3 -0
  8. Einstein-v5-v0.2-7B-Q3_K_M.gguf +3 -0
  9. Einstein-v5-v0.2-7B-Q3_K_S.gguf +3 -0
  10. Einstein-v5-v0.2-7B-Q4_0.gguf +3 -0
  11. Einstein-v5-v0.2-7B-Q4_K_M.gguf +3 -0
  12. Einstein-v5-v0.2-7B-Q4_K_S.gguf +3 -0
  13. Einstein-v5-v0.2-7B-Q5_0.gguf +3 -0
  14. Einstein-v5-v0.2-7B-Q5_K_M.gguf +3 -0
  15. Einstein-v5-v0.2-7B-Q5_K_S.gguf +3 -0
  16. Einstein-v5-v0.2-7B-Q6_K.gguf +3 -0
  17. Einstein-v5-v0.2-7B-Q8_0.gguf +3 -0
  18. README.md +84 -0
.gitattributes CHANGED
@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Einstein-v5-v0.2-7B-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Einstein-v5-v0.2-7B-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Einstein-v5-v0.2-7B-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Einstein-v5-v0.2-7B-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Einstein-v5-v0.2-7B-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Einstein-v5-v0.2-7B-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Einstein-v5-v0.2-7B-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Einstein-v5-v0.2-7B-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Einstein-v5-v0.2-7B-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Einstein-v5-v0.2-7B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Einstein-v5-v0.2-7B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Einstein-v5-v0.2-7B-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Einstein-v5-v0.2-7B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Einstein-v5-v0.2-7B-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Einstein-v5-v0.2-7B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Einstein-v5-v0.2-7B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Einstein-v5-v0.2-7B-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:777473d7b45e1ecc10a3b1b4cb1d8a82f1a6d742e48c46731ca5438caa86852c
3
+ size 3284902304
Einstein-v5-v0.2-7B-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1ffdf891c61dbfb151ecd0a872de1799f5385cd81a16d2fe95872ffdb75b5b3
3
+ size 3182404000
Einstein-v5-v0.2-7B-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f40048b5386db01d8724dfc782aa1eb195a25af51ce05f60cd2f77a830312505
3
+ size 4155065824
Einstein-v5-v0.2-7B-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:957db5138bc92d59bd91b5296af74ebe5b85b84732d46aa731287e0695780f45
3
+ size 3944400096
Einstein-v5-v0.2-7B-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b571f6621825530db5778696058b7767d74f4c8df6ac0b04eb9ebd6feec075b3
3
+ size 2719252064
Einstein-v5-v0.2-7B-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b75af80f53cb134f92d4bede5d2d5f31945e25a80940abfa2897e6e181b66ee
3
+ size 3822035360
Einstein-v5-v0.2-7B-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a6d46579e9f759a71fe85b45b89c1da3c13eaa0d02bc3db57c1712d61999f0ee
3
+ size 3518996896
Einstein-v5-v0.2-7B-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:43c6fc5fcec1de5e401d40945e046c5b61898aa1252f92a4a057440b49d2ac62
3
+ size 3164578208
Einstein-v5-v0.2-7B-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:116b3fbbcb72276a3205f1cf6ebb43c2c023adc6a8dbf3bc5283dece86ee6b94
3
+ size 4108928480
Einstein-v5-v0.2-7B-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6d69b2a32c3ed281ecd1494c78a89f679e32ff4a8d81699dc8ad150e5fec96e1
3
+ size 4368451040
Einstein-v5-v0.2-7B-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da76d7bc0d68b43df7ff160388ffccdb394d537b854dff8a2ec0121d622b6c62
3
+ size 4140385760
Einstein-v5-v0.2-7B-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ca0ccd15d49697d2cf64caf6fa15b8ebb6a9d353304b92d5fa397b02fa33f06c
3
+ size 4997728736
Einstein-v5-v0.2-7B-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:048d2a8f38b52a257766578f3b690e72bf3139b794f6225b18a8716a1eb865bb
3
+ size 5131422176
Einstein-v5-v0.2-7B-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee3dc7e377aea6171ea4d2b90f0878fd80592868625f7c0289ca78d4448fd6c0
3
+ size 4997728736
Einstein-v5-v0.2-7B-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6513960a38d4aa94ee08bfff0b84eae0793fafca417e9e34a57fe864293b7d5
3
+ size 5942079008
Einstein-v5-v0.2-7B-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a79f748c68252758df48a1b2b738ca5dca84dfee314ed00a609b7badc00d55b6
3
+ size 7695875488
README.md ADDED
@@ -0,0 +1,84 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ tags:
4
+ - axolotl
5
+ - generated_from_trainer
6
+ - Mistral
7
+ - instruct
8
+ - finetune
9
+ - chatml
10
+ - gpt4
11
+ - synthetic data
12
+ - science
13
+ - physics
14
+ - chemistry
15
+ - biology
16
+ - math
17
+ base_model: alpindale/Mistral-7B-v0.2-hf
18
+ datasets:
19
+ - allenai/ai2_arc
20
+ - camel-ai/physics
21
+ - camel-ai/chemistry
22
+ - camel-ai/biology
23
+ - camel-ai/math
24
+ - metaeval/reclor
25
+ - openbookqa
26
+ - mandyyyyii/scibench
27
+ - derek-thomas/ScienceQA
28
+ - TIGER-Lab/ScienceEval
29
+ - jondurbin/airoboros-3.2
30
+ - LDJnr/Capybara
31
+ - Cot-Alpaca-GPT4-From-OpenHermes-2.5
32
+ - STEM-AI-mtl/Electrical-engineering
33
+ - knowrohit07/saraswati-stem
34
+ - sablo/oasst2_curated
35
+ - lmsys/lmsys-chat-1m
36
+ - TIGER-Lab/MathInstruct
37
+ - bigbio/med_qa
38
+ - meta-math/MetaMathQA-40K
39
+ - openbookqa
40
+ - piqa
41
+ - metaeval/reclor
42
+ - derek-thomas/ScienceQA
43
+ - scibench
44
+ - sciq
45
+ - Open-Orca/SlimOrca
46
+ - migtissera/Synthia-v1.3
47
+ - TIGER-Lab/ScienceEval
48
+ - allenai/WildChat
49
+ - microsoft/orca-math-word-problems-200k
50
+ - openchat/openchat_sharegpt4_dataset
51
+ - teknium/GPTeacher-General-Instruct
52
+ - m-a-p/CodeFeedback-Filtered-Instruction
53
+ quantized_by: bartowski
54
+ pipeline_tag: text-generation
55
+ ---
56
+
57
+ ## Llamacpp Quantizations of Einstein-v5-v0.2-7B
58
+
59
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2536">b2536</a> for quantization.
60
+
61
+ Original model: https://huggingface.co/Weyaxi/Einstein-v5-v0.2-7B
62
+
63
+ Download a file (not the whole branch) from below:
64
+
65
+ | Filename | Quant type | File Size | Description |
66
+ | -------- | ---------- | --------- | ----------- |
67
+ | [Einstein-v5-v0.2-7B-Q8_0.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q8_0.gguf) | Q8_0 | 7.69GB | Extremely high quality, generally unneeded but max available quant. |
68
+ | [Einstein-v5-v0.2-7B-Q6_K.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q6_K.gguf) | Q6_K | 5.94GB | Very high quality, near perfect, *recommended*. |
69
+ | [Einstein-v5-v0.2-7B-Q5_K_M.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q5_K_M.gguf) | Q5_K_M | 5.13GB | High quality, very usable. |
70
+ | [Einstein-v5-v0.2-7B-Q5_K_S.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q5_K_S.gguf) | Q5_K_S | 4.99GB | High quality, very usable. |
71
+ | [Einstein-v5-v0.2-7B-Q5_0.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q5_0.gguf) | Q5_0 | 4.99GB | High quality, older format, generally not recommended. |
72
+ | [Einstein-v5-v0.2-7B-Q4_K_M.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q4_K_M.gguf) | Q4_K_M | 4.36GB | Good quality, uses about 4.83 bits per weight. |
73
+ | [Einstein-v5-v0.2-7B-Q4_K_S.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q4_K_S.gguf) | Q4_K_S | 4.14GB | Slightly lower quality with small space savings. |
74
+ | [Einstein-v5-v0.2-7B-IQ4_NL.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-IQ4_NL.gguf) | IQ4_NL | 4.15GB | Decent quality, similar to Q4_K_S, new method of quanting, |
75
+ | [Einstein-v5-v0.2-7B-IQ4_XS.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-IQ4_XS.gguf) | IQ4_XS | 3.94GB | Decent quality, new method with similar performance to Q4. |
76
+ | [Einstein-v5-v0.2-7B-Q4_0.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q4_0.gguf) | Q4_0 | 4.10GB | Decent quality, older format, generally not recommended. |
77
+ | [Einstein-v5-v0.2-7B-Q3_K_L.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q3_K_L.gguf) | Q3_K_L | 3.82GB | Lower quality but usable, good for low RAM availability. |
78
+ | [Einstein-v5-v0.2-7B-Q3_K_M.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q3_K_M.gguf) | Q3_K_M | 3.51GB | Even lower quality. |
79
+ | [Einstein-v5-v0.2-7B-IQ3_M.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-IQ3_M.gguf) | IQ3_M | 3.28GB | Medium-low quality, new method with decent performance. |
80
+ | [Einstein-v5-v0.2-7B-IQ3_S.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-IQ3_S.gguf) | IQ3_S | 3.18GB | Lower quality, new method with decent performance, recommended over Q3 quants. |
81
+ | [Einstein-v5-v0.2-7B-Q3_K_S.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q3_K_S.gguf) | Q3_K_S | 3.16GB | Low quality, not recommended. |
82
+ | [Einstein-v5-v0.2-7B-Q2_K.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q2_K.gguf) | Q2_K | 2.71GB | Extremely low quality, *not* recommended.
83
+
84
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski