asedmammad
commited on
Commit
•
95ca2c0
1
Parent(s):
87d0f1f
Update README.md
Browse files
README.md
CHANGED
@@ -34,6 +34,12 @@ Change `-ngl 32` to the number of layers to offload to GPU. Remove it if you don
|
|
34 |
|
35 |
If you want to have a chat-style conversation, replace the `-p <PROMPT>` argument with `-i -ins`, you can use `--interactive-first` to start in interactive mode.
|
36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
37 |
## Compatibility
|
38 |
|
39 |
I have uploded both the original llama.cpp quant methods (`q4_0, q4_1, q5_0, q5_1, q8_0`) as well as the k-quant methods (`q2_K, q3_K_S, q3_K_M, q3_K_L, q4_K_S, q4_K_M, q5_K_S, q6_K`).
|
|
|
34 |
|
35 |
If you want to have a chat-style conversation, replace the `-p <PROMPT>` argument with `-i -ins`, you can use `--interactive-first` to start in interactive mode.
|
36 |
|
37 |
+
Though the model outputs good persian text among the open source models, enabling sampling or using high temperature values in llama.cpp causes the model to output non sense persian text, so currently I am using it with low temperature value with alpaca instruct template:
|
38 |
+
|
39 |
+
```
|
40 |
+
./main -t 2 -ngl 32 -m PersianMind-v1.0.q4_K_M.gguf --color -c 2048 --temp 0.2 --repeat_penalty 1.2 -n -1 -e -p "### Instruction: در مورد زنبور عسل توضیح بده ### Response:"
|
41 |
+
```
|
42 |
+
|
43 |
## Compatibility
|
44 |
|
45 |
I have uploded both the original llama.cpp quant methods (`q4_0, q4_1, q5_0, q5_1, q8_0`) as well as the k-quant methods (`q2_K, q3_K_S, q3_K_M, q3_K_L, q4_K_S, q4_K_M, q5_K_S, q6_K`).
|