Spaces:
Running
on
Zero
Running
on
Zero
File size: 679 Bytes
886d8e9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
---
title: vLLM
---
To use Open Interpreter with vLLM, you will need to:
1. `pip install vllm`
2. Set the api_base flag:
<CodeGroup>
```bash Terminal
interpreter --api_base <https://your-hosted-vllm-server>
```
```python Python
from interpreter import interpreter
interpreter.llm.api_base = "<https://your-hosted-vllm-server>"
interpreter.chat()
```
</CodeGroup>
3. Set the `model` flag:
<CodeGroup>
```bash Terminal
interpreter --model vllm/<perplexity-model>
```
```python Python
from interpreter import interpreter
interpreter.llm.model = "vllm/<perplexity-model>"
interpreter.chat()
```
</CodeGroup>
# Supported Models
All models from VLLM should be supported
|