Update README.md
Browse files
README.md
CHANGED
@@ -49,6 +49,26 @@ These new methods were released to llama.cpp on 26th April. You will need to pul
|
|
49 |
|
50 |
Don't expect any third-party UIs/tools to support them yet.
|
51 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
52 |
# Original model info
|
53 |
|
54 |
Overview of Evol-Instruct
|
|
|
49 |
|
50 |
Don't expect any third-party UIs/tools to support them yet.
|
51 |
|
52 |
+
## How to run in `llama.cpp`
|
53 |
+
|
54 |
+
I use the following command line; adjust for your tastes and needs:
|
55 |
+
|
56 |
+
```
|
57 |
+
./main -t 18 -m WizardLM-7B.GGML.q4_2.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "Below is an instruction that describes a task. Write a response that appropriately completes the request.
|
58 |
+
### Instruction:
|
59 |
+
Write a story about llamas
|
60 |
+
### Response:"
|
61 |
+
```
|
62 |
+
Change `-t 18` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
|
63 |
+
|
64 |
+
If you want to have a chat-style conversation, replace the `-p <PROMPT>` argument with `-i -ins`
|
65 |
+
|
66 |
+
## How to run in `text-generation-webui`
|
67 |
+
|
68 |
+
Put the desired .bin file in a model directory with `ggml` (case sensitive) in its name.
|
69 |
+
|
70 |
+
Further instructions here: [text-generation-webui/docs/llama.cpp-models.md](https://github.com/oobabooga/text-generation-webui/blob/main/docs/llama.cpp-models.md).
|
71 |
+
|
72 |
# Original model info
|
73 |
|
74 |
Overview of Evol-Instruct
|