Upload README.md
Browse files
README.md
CHANGED
@@ -94,7 +94,7 @@ Documentation on installing and using vLLM [can be found here](https://vllm.read
|
|
94 |
- When using vLLM as a server, pass the `--quantization awq` parameter, for example:
|
95 |
|
96 |
```shell
|
97 |
-
python3 python -m vllm.entrypoints.api_server --model TheBloke/Llama-2-7B-
|
98 |
```
|
99 |
|
100 |
When using vLLM from Python code, pass the `quantization=awq` parameter, for example:
|
|
|
94 |
- When using vLLM as a server, pass the `--quantization awq` parameter, for example:
|
95 |
|
96 |
```shell
|
97 |
+
python3 python -m vllm.entrypoints.api_server --model TheBloke/Llama-2-7B-AWQ --quantization awq
|
98 |
```
|
99 |
|
100 |
When using vLLM from Python code, pass the `quantization=awq` parameter, for example:
|