TheBloke commited on
Commit
efc44ea
1 Parent(s): b83783d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -0
README.md CHANGED
@@ -49,6 +49,26 @@ These new methods were released to llama.cpp on 26th April. You will need to pul
49
 
50
  Don't expect any third-party UIs/tools to support them yet.
51
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
  # Original model info
53
 
54
  Overview of Evol-Instruct
 
49
 
50
  Don't expect any third-party UIs/tools to support them yet.
51
 
52
+ ## How to run in `llama.cpp`
53
+
54
+ I use the following command line; adjust for your tastes and needs:
55
+
56
+ ```
57
+ ./main -t 18 -m WizardLM-7B.GGML.q4_2.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "Below is an instruction that describes a task. Write a response that appropriately completes the request.
58
+ ### Instruction:
59
+ Write a story about llamas
60
+ ### Response:"
61
+ ```
62
+ Change `-t 18` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
63
+
64
+ If you want to have a chat-style conversation, replace the `-p <PROMPT>` argument with `-i -ins`
65
+
66
+ ## How to run in `text-generation-webui`
67
+
68
+ Put the desired .bin file in a model directory with `ggml` (case sensitive) in its name.
69
+
70
+ Further instructions here: [text-generation-webui/docs/llama.cpp-models.md](https://github.com/oobabooga/text-generation-webui/blob/main/docs/llama.cpp-models.md).
71
+
72
  # Original model info
73
 
74
  Overview of Evol-Instruct