Hugging Face's logo

legraphista
/
Phi-3-mini-4k-instruct-IMat-GGUF

Text Generation
GGUF
English
quantized
GGUF
imatrix
quantization
imat
static
conversational
Model card Files Community
legraphista commited on
Commit
4be0971
•
1 Parent(s): 5e95862

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +85 -35
README.md CHANGED
@@ -13,72 +13,118 @@ tags:
13
  - GGUF
14
  - imatrix
15
  - quantization
 
 
 
16
  ---
17
 
18
- # Phi-3-mini-4k-instruct-GGUF
19
- _Llama.cpp imatrix quantization of Phi-3-mini-4k-instruct-GGUF_
20
 
21
  Original Model: [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)
22
  Original dtype: `BF16` (`bfloat16`)
23
- Quantized by: llama.cpp [b2989](https://github.com/ggerganov/llama.cpp/releases/tag/b2989)
24
  IMatrix dataset: [here](https://gist.githubusercontent.com/legraphista/d6d93f1a254bcfc58e0af3777eaec41e/raw/d380e7002cea4a51c33fffd47db851942754e7cc/imatrix.calibration.medium.raw)
25
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  ## Files
27
 
28
  ### IMatrix
29
  Status: ✅ Available
30
- Link: [here](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/imatrix.dat)
31
 
32
  ### Common Quants
33
  | Filename | Quant type | File Size | Status | Uses IMatrix | Is Split |
34
  | -------- | ---------- | --------- | ------ | ------------ | -------- |
35
- | [Phi-3-mini-4k-instruct.Q8_0.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.Q8_0.gguf) | Q8_0 | 4.06GB | ✅ Available | No | 📦 No
36
- | [Phi-3-mini-4k-instruct.Q6_K.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.Q6_K.gguf) | Q6_K | 3.14GB | ✅ Available | No | 📦 No
37
- | [Phi-3-mini-4k-instruct.Q4_K.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.Q4_K.gguf) | Q4_K | 2.39GB | ✅ Available | Yes | 📦 No
38
- | [Phi-3-mini-4k-instruct.Q3_K.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.Q3_K.gguf) | Q3_K | 1.96GB | ✅ Available | Yes | 📦 No
39
- | [Phi-3-mini-4k-instruct.Q2_K.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.Q2_K.gguf) | Q2_K | 1.42GB | ✅ Available | Yes | 📦 No
40
 
41
 
42
  ### All Quants
43
  | Filename | Quant type | File Size | Status | Uses IMatrix | Is Split |
44
  | -------- | ---------- | --------- | ------ | ------------ | -------- |
45
- | [Phi-3-mini-4k-instruct.FP16.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.FP16.gguf) | F16 | 7.64GB | ✅ Available | No | 📦 No
46
- | [Phi-3-mini-4k-instruct.BF16.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.BF16.gguf) | BF16 | 7.64GB | ✅ Available | No | 📦 No
47
- | [Phi-3-mini-4k-instruct.Q5_K.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.Q5_K.gguf) | Q5_K | 2.82GB | ✅ Available | No | 📦 No
48
- | [Phi-3-mini-4k-instruct.Q5_K_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.Q5_K_S.gguf) | Q5_K_S | 2.64GB | ✅ Available | No | 📦 No
49
- | [Phi-3-mini-4k-instruct.Q4_K_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.Q4_K_S.gguf) | Q4_K_S | 2.19GB | ✅ Available | Yes | 📦 No
50
- | [Phi-3-mini-4k-instruct.Q3_K_L.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.Q3_K_L.gguf) | Q3_K_L | 2.09GB | ✅ Available | Yes | 📦 No
51
- | [Phi-3-mini-4k-instruct.Q3_K_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.Q3_K_S.gguf) | Q3_K_S | 1.68GB | ✅ Available | Yes | 📦 No
52
- | [Phi-3-mini-4k-instruct.Q2_K_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.Q2_K_S.gguf) | Q2_K_S | 1.33GB | ✅ Available | Yes | 📦 No
53
- | [Phi-3-mini-4k-instruct.IQ4_NL.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ4_NL.gguf) | IQ4_NL | 2.18GB | ✅ Available | Yes | 📦 No
54
- | [Phi-3-mini-4k-instruct.IQ4_XS.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ4_XS.gguf) | IQ4_XS | 2.06GB | ✅ Available | Yes | 📦 No
55
- | [Phi-3-mini-4k-instruct.IQ3_M.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ3_M.gguf) | IQ3_M | 1.86GB | ✅ Available | Yes | 📦 No
56
- | [Phi-3-mini-4k-instruct.IQ3_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ3_S.gguf) | IQ3_S | 1.68GB | ✅ Available | Yes | 📦 No
57
- | [Phi-3-mini-4k-instruct.IQ3_XS.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ3_XS.gguf) | IQ3_XS | 1.63GB | ✅ Available | Yes | 📦 No
58
- | [Phi-3-mini-4k-instruct.IQ3_XXS.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ3_XXS.gguf) | IQ3_XXS | 1.51GB | ✅ Available | Yes | 📦 No
59
- | [Phi-3-mini-4k-instruct.IQ2_M.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ2_M.gguf) | IQ2_M | 1.32GB | ✅ Available | Yes | 📦 No
60
- | [Phi-3-mini-4k-instruct.IQ2_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ2_S.gguf) | IQ2_S | 1.22GB | ✅ Available | Yes | 📦 No
61
- | [Phi-3-mini-4k-instruct.IQ2_XS.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ2_XS.gguf) | IQ2_XS | 1.15GB | ✅ Available | Yes | 📦 No
62
- | [Phi-3-mini-4k-instruct.IQ2_XXS.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ2_XXS.gguf) | IQ2_XXS | 1.04GB | ✅ Available | Yes | 📦 No
63
- | [Phi-3-mini-4k-instruct.IQ1_M.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ1_M.gguf) | IQ1_M | 917.11MB | ✅ Available | Yes | 📦 No
64
- | [Phi-3-mini-4k-instruct.IQ1_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ1_S.gguf) | IQ1_S | 841.61MB | ✅ Available | Yes | 📦 No
65
 
66
 
67
  ## Downloading using huggingface-cli
68
- First, make sure you have hugginface-cli installed:
69
  ```
70
  pip install -U "huggingface_hub[cli]"
71
  ```
72
- Then, you can target the specific file you want:
73
  ```
74
- huggingface-cli download legraphista/Phi-3-mini-4k-instruct-GGUF --include "Phi-3-mini-4k-instruct.Q8_0.gguf" --local-dir ./
75
  ```
76
- If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
77
  ```
78
- huggingface-cli download legraphista/Phi-3-mini-4k-instruct-GGUF --include "Phi-3-mini-4k-instruct.Q8_0/*" --local-dir Phi-3-mini-4k-instruct.Q8_0
79
  # see FAQ for merging GGUF's
80
  ```
81
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
82
  ## FAQ
83
 
84
  ### Why is the IMatrix not applied everywhere?
@@ -91,4 +137,8 @@ According to [this investigation](https://www.reddit.com/r/LocalLLaMA/comments/1
91
  - Unzip the archive and you should be able to find `gguf-split`
92
  2. Locate your GGUF chunks folder (ex: `Phi-3-mini-4k-instruct.Q8_0`)
93
  3. Run `gguf-split --merge Phi-3-mini-4k-instruct.Q8_0/Phi-3-mini-4k-instruct.Q8_0-00001-of-XXXXX.gguf Phi-3-mini-4k-instruct.Q8_0.gguf`
94
- - Make sure to point `gguf-split` to the first chunk of the split.
 
 
 
 
 
13
  - GGUF
14
  - imatrix
15
  - quantization
16
+ - imat
17
+ - imatrix
18
+ - static
19
  ---
20
 
21
+ # Phi-3-mini-4k-instruct-IMat-GGUF
22
+ _Llama.cpp imatrix quantization of microsoft/Phi-3-mini-4k-instruct_
23
 
24
  Original Model: [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)
25
  Original dtype: `BF16` (`bfloat16`)
26
+ Quantized by: llama.cpp [b3001](https://github.com/ggerganov/llama.cpp/releases/tag/b3001)
27
  IMatrix dataset: [here](https://gist.githubusercontent.com/legraphista/d6d93f1a254bcfc58e0af3777eaec41e/raw/d380e7002cea4a51c33fffd47db851942754e7cc/imatrix.calibration.medium.raw)
28
 
29
+ - [Phi-3-mini-4k-instruct-IMat-GGUF](#phi-3-mini-4k-instruct-imat-gguf)
30
+ - [Files](#files)
31
+ - [IMatrix](#imatrix)
32
+ - [Common Quants](#common-quants)
33
+ - [All Quants](#all-quants)
34
+ - [Downloading using huggingface-cli](#downloading-using-huggingface-cli)
35
+ - [Inference](#inference)
36
+ - [Simple chat template](#simple-chat-template)
37
+ - [Llama.cpp](#llama-cpp)
38
+ - [FAQ](#faq)
39
+ - [Why is the IMatrix not applied everywhere?](#why-is-the-imatrix-not-applied-everywhere)
40
+ - [How do I merge a split GGUF?](#how-do-i-merge-a-split-gguf)
41
+
42
+ ---
43
+
44
  ## Files
45
 
46
  ### IMatrix
47
  Status: ✅ Available
48
+ Link: [here](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/imatrix.dat)
49
 
50
  ### Common Quants
51
  | Filename | Quant type | File Size | Status | Uses IMatrix | Is Split |
52
  | -------- | ---------- | --------- | ------ | ------------ | -------- |
53
+ | [Phi-3-mini-4k-instruct.Q8_0.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.Q8_0.gguf) | Q8_0 | 4.06GB | ✅ Available | ⚪ No | 📦 No
54
+ | [Phi-3-mini-4k-instruct.Q6_K.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.Q6_K.gguf) | Q6_K | 3.14GB | ✅ Available | ⚪ No | 📦 No
55
+ | [Phi-3-mini-4k-instruct.Q4_K.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.Q4_K.gguf) | Q4_K | 2.39GB | ✅ Available | 🟢 Yes | 📦 No
56
+ | [Phi-3-mini-4k-instruct.Q3_K.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.Q3_K.gguf) | Q3_K | 1.96GB | ✅ Available | 🟢 Yes | 📦 No
57
+ | [Phi-3-mini-4k-instruct.Q2_K.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.Q2_K.gguf) | Q2_K | 1.42GB | ✅ Available | 🟢 Yes | 📦 No
58
 
59
 
60
  ### All Quants
61
  | Filename | Quant type | File Size | Status | Uses IMatrix | Is Split |
62
  | -------- | ---------- | --------- | ------ | ------------ | -------- |
63
+ | [Phi-3-mini-4k-instruct.FP16.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.FP16.gguf) | F16 | 7.64GB | ✅ Available | ⚪ No | 📦 No
64
+ | [Phi-3-mini-4k-instruct.BF16.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.BF16.gguf) | BF16 | 7.64GB | ✅ Available | ⚪ No | 📦 No
65
+ | [Phi-3-mini-4k-instruct.Q5_K.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.Q5_K.gguf) | Q5_K | 2.82GB | ✅ Available | ⚪ No | 📦 No
66
+ | [Phi-3-mini-4k-instruct.Q5_K_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.Q5_K_S.gguf) | Q5_K_S | 2.64GB | ✅ Available | ⚪ No | 📦 No
67
+ | [Phi-3-mini-4k-instruct.Q4_K_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.Q4_K_S.gguf) | Q4_K_S | 2.19GB | ✅ Available | 🟢 Yes | 📦 No
68
+ | [Phi-3-mini-4k-instruct.Q3_K_L.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.Q3_K_L.gguf) | Q3_K_L | 2.09GB | ✅ Available | 🟢 Yes | 📦 No
69
+ | [Phi-3-mini-4k-instruct.Q3_K_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.Q3_K_S.gguf) | Q3_K_S | 1.68GB | ✅ Available | 🟢 Yes | 📦 No
70
+ | [Phi-3-mini-4k-instruct.Q2_K_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.Q2_K_S.gguf) | Q2_K_S | 1.33GB | ✅ Available | 🟢 Yes | 📦 No
71
+ | [Phi-3-mini-4k-instruct.IQ4_NL.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ4_NL.gguf) | IQ4_NL | 2.18GB | ✅ Available | 🟢 Yes | 📦 No
72
+ | [Phi-3-mini-4k-instruct.IQ4_XS.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ4_XS.gguf) | IQ4_XS | 2.06GB | ✅ Available | 🟢 Yes | 📦 No
73
+ | [Phi-3-mini-4k-instruct.IQ3_M.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ3_M.gguf) | IQ3_M | 1.86GB | ✅ Available | 🟢 Yes | 📦 No
74
+ | [Phi-3-mini-4k-instruct.IQ3_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ3_S.gguf) | IQ3_S | 1.68GB | ✅ Available | 🟢 Yes | 📦 No
75
+ | [Phi-3-mini-4k-instruct.IQ3_XS.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ3_XS.gguf) | IQ3_XS | 1.63GB | ✅ Available | 🟢 Yes | 📦 No
76
+ | [Phi-3-mini-4k-instruct.IQ3_XXS.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ3_XXS.gguf) | IQ3_XXS | 1.51GB | ✅ Available | 🟢 Yes | 📦 No
77
+ | [Phi-3-mini-4k-instruct.IQ2_M.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ2_M.gguf) | IQ2_M | 1.32GB | ✅ Available | 🟢 Yes | 📦 No
78
+ | [Phi-3-mini-4k-instruct.IQ2_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ2_S.gguf) | IQ2_S | 1.22GB | ✅ Available | 🟢 Yes | 📦 No
79
+ | [Phi-3-mini-4k-instruct.IQ2_XS.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ2_XS.gguf) | IQ2_XS | 1.15GB | ✅ Available | 🟢 Yes | 📦 No
80
+ | [Phi-3-mini-4k-instruct.IQ2_XXS.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ2_XXS.gguf) | IQ2_XXS | 1.04GB | ✅ Available | 🟢 Yes | 📦 No
81
+ | [Phi-3-mini-4k-instruct.IQ1_M.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ1_M.gguf) | IQ1_M | 917.11MB | ✅ Available | 🟢 Yes | 📦 No
82
+ | [Phi-3-mini-4k-instruct.IQ1_S.gguf](https://huggingface.co/legraphista/Phi-3-mini-4k-instruct-IMat-GGUF/blob/main/Phi-3-mini-4k-instruct.IQ1_S.gguf) | IQ1_S | 841.61MB | ✅ Available | 🟢 Yes | 📦 No
83
 
84
 
85
  ## Downloading using huggingface-cli
86
+ If you do not have hugginface-cli installed:
87
  ```
88
  pip install -U "huggingface_hub[cli]"
89
  ```
90
+ Download the specific file you want:
91
  ```
92
+ huggingface-cli download legraphista/Phi-3-mini-4k-instruct-IMat-GGUF --include "Phi-3-mini-4k-instruct.Q8_0.gguf" --local-dir ./
93
  ```
94
+ If the model file is big, it has been split into multiple files. In order to download them all to a local folder, run:
95
  ```
96
+ huggingface-cli download legraphista/Phi-3-mini-4k-instruct-IMat-GGUF --include "Phi-3-mini-4k-instruct.Q8_0/*" --local-dir Phi-3-mini-4k-instruct.Q8_0
97
  # see FAQ for merging GGUF's
98
  ```
99
 
100
+ ---
101
+
102
+ ## Inference
103
+
104
+ ### Simple chat template
105
+ ```
106
+ <|user|>
107
+ I am going to Paris, what should I see?<|end|>
108
+ <|assistant|>
109
+ Paris, the capital of France, is known for its stunning architecture, art museums, historical landmarks, and romantic atmosphere. Here are some of the top attractions to see in Paris:
110
+
111
+ 1. The Eiffel Tower: The iconic Eiffel Tower is one of the most recognizable landmarks in the world and offers breathtaking views of the city.
112
+ 2. The Louvre Museum: The Louvre is one of the world's largest and most famous museums, housing an impressive collection of art and artifacts, including the Mona Lisa.
113
+ 3. Notre-Dame Cathedral: This beautiful cathedral is one of the most famous landmarks in Paris and is known for its Gothic architecture and stunning stained glass windows.
114
+
115
+ These are just a few of the many attractions that Paris has to offer. With so much to see and do, it's no wonder that Paris is one of the most popular tourist destinations in the world."<|end|>
116
+ <|user|>
117
+ What is so great about #1?<|end|>
118
+ <|assistant|>
119
+ ```
120
+
121
+ ### Llama.cpp
122
+ ```
123
+ llama.cpp/main -m Phi-3-mini-4k-instruct.Q8_0.gguf --color -i -p "prompt here (according to the chat template)"
124
+ ```
125
+
126
+ ---
127
+
128
  ## FAQ
129
 
130
  ### Why is the IMatrix not applied everywhere?
 
137
  - Unzip the archive and you should be able to find `gguf-split`
138
  2. Locate your GGUF chunks folder (ex: `Phi-3-mini-4k-instruct.Q8_0`)
139
  3. Run `gguf-split --merge Phi-3-mini-4k-instruct.Q8_0/Phi-3-mini-4k-instruct.Q8_0-00001-of-XXXXX.gguf Phi-3-mini-4k-instruct.Q8_0.gguf`
140
+ - Make sure to point `gguf-split` to the first chunk of the split.
141
+
142
+ ---
143
+
144
+ Got a suggestion? Ping me [@legraphista](https://x.com/legraphista)!