Hugging Face's logo

SanctumAI
/
Phi-3.5-mini-instruct-GGUF

Text Generation
GGUF
multilingual
phi3
nlp
code
Inference Endpoints
conversational
Model card Files Community
AndreyBest commited on
Commit
68677c9
•
1 Parent(s): ccb1438

Reorder items in requirements table.

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -46,14 +46,14 @@ Prompt template:
46
  | [phi-3.5-mini-instruct.Q3_K_M.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q3_K_M.gguf) | Q3_K_M | 1.96 GB | 5.52 GB |
47
  | [phi-3.5-mini-instruct.Q3_K_L.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q3_K_L.gguf) | Q3_K_L | 2.09 GB | 5.65 GB |
48
  | [phi-3.5-mini-instruct.Q4_0.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_0.gguf) | Q4_0 | 2.18 GB | 5.73 GB |
49
- | [phi-3.5-mini-instruct.Q4_K_S.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_K_S.gguf) | Q4_K_S | 2.19 GB | 5.74 GB |
50
- | [phi-3.5-mini-instruct.Q4_K.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_K.gguf) | Q4_K | 2.39 GB | 5.93 GB |
51
- | [phi-3.5-mini-instruct.Q4_K_M.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_K_M.gguf) | Q4_K_M | 2.39 GB | 5.93 GB |
52
  | [phi-3.5-mini-instruct.Q4_1.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_1.gguf) | Q4_1 | 2.41 GB | 5.94 GB |
53
  | [phi-3.5-mini-instruct.Q5_0.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_0.gguf) | Q5_0 | 2.64 GB | 6.16 GB |
54
- | [phi-3.5-mini-instruct.Q5_K_S.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_K_S.gguf) | Q5_K_S | 2.64 GB | 6.16 GB |
55
- | [phi-3.5-mini-instruct.Q5_K.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_K.gguf) | Q5_K | 2.82 GB | 6.32 GB |
56
- | [phi-3.5-mini-instruct.Q5_K_M.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_K_M.gguf) | Q5_K_M | 2.82 GB | 6.32 GB |
57
  | [phi-3.5-mini-instruct.Q5_1.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_1.gguf) | Q5_1 | 2.87 GB | 6.38 GB |
58
  | [phi-3.5-mini-instruct.Q6_K.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q6_K.gguf) | Q6_K | 3.14 GB | 6.62 GB |
59
  | [phi-3.5-mini-instruct.Q8_0.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q8_0.gguf) | Q8_0 | 4.06 GB | 7.48 GB |
 
46
  | [phi-3.5-mini-instruct.Q3_K_M.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q3_K_M.gguf) | Q3_K_M | 1.96 GB | 5.52 GB |
47
  | [phi-3.5-mini-instruct.Q3_K_L.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q3_K_L.gguf) | Q3_K_L | 2.09 GB | 5.65 GB |
48
  | [phi-3.5-mini-instruct.Q4_0.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_0.gguf) | Q4_0 | 2.18 GB | 5.73 GB |
49
+ | [phi-3.5-mini-instruct.Q4_K_S.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_K_S.gguf) | Q4_K_S | 2.19 GB | 5.74 GB |
50
+ | [phi-3.5-mini-instruct.Q4_K_M.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_K_M.gguf) | Q4_K_M | 2.39 GB | 5.93 GB |
51
+ | [phi-3.5-mini-instruct.Q4_K.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_K.gguf) | Q4_K | 2.39 GB | 5.93 GB |
52
  | [phi-3.5-mini-instruct.Q4_1.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_1.gguf) | Q4_1 | 2.41 GB | 5.94 GB |
53
  | [phi-3.5-mini-instruct.Q5_0.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_0.gguf) | Q5_0 | 2.64 GB | 6.16 GB |
54
+ | [phi-3.5-mini-instruct.Q5_K_S.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_K_S.gguf) | Q5_K_S | 2.64 GB | 6.16 GB |
55
+ | [phi-3.5-mini-instruct.Q5_K_M.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_K_M.gguf) | Q5_K_M | 2.82 GB | 6.32 GB |
56
+ | [phi-3.5-mini-instruct.Q5_K.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_K.gguf) | Q5_K | 2.82 GB | 6.32 GB |
57
  | [phi-3.5-mini-instruct.Q5_1.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_1.gguf) | Q5_1 | 2.87 GB | 6.38 GB |
58
  | [phi-3.5-mini-instruct.Q6_K.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q6_K.gguf) | Q6_K | 3.14 GB | 6.62 GB |
59
  | [phi-3.5-mini-instruct.Q8_0.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q8_0.gguf) | Q8_0 | 4.06 GB | 7.48 GB |