Spaces:
Running
on
Zero
Running
on
Zero
File size: 6,552 Bytes
886d8e9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 |
---
title: Streaming Response
---
You can stream messages, code, and code outputs out of Open Interpreter by setting `stream=True` in an `interpreter.chat(message)` call.
```python
for chunk in interpreter.chat("What's 34/24?", stream=True, display=False):
print(chunk)
```
```
{"role": "assistant", "type": "code", "format": "python", "start": True}
{"role": "assistant", "type": "code", "format": "python", "content": "34"}
{"role": "assistant", "type": "code", "format": "python", "content": " /"}
{"role": "assistant", "type": "code", "format": "python", "content": " "}
{"role": "assistant", "type": "code", "format": "python", "content": "24"}
{"role": "assistant", "type": "code", "format": "python", "end": True}
{"role": "computer", "type": "confirmation", "format": "execution", "content": {"type": "code", "format": "python", "content": "34 / 24"}},
{"role": "computer", "type": "console", "start": True}
{"role": "computer", "type": "console", "format": "active_line", "content": "1"}
{"role": "computer", "type": "console", "format": "output", "content": "1.4166666666666667\n"}
{"role": "computer", "type": "console", "format": "active_line", "content": None},
{"role": "computer", "type": "console", "end": True}
{"role": "assistant", "type": "message", "start": True}
{"role": "assistant", "type": "message", "content": "The"}
{"role": "assistant", "type": "message", "content": " result"}
{"role": "assistant", "type": "message", "content": " of"}
{"role": "assistant", "type": "message", "content": " the"}
{"role": "assistant", "type": "message", "content": " division"}
{"role": "assistant", "type": "message", "content": " "}
{"role": "assistant", "type": "message", "content": "34"}
{"role": "assistant", "type": "message", "content": "/"}
{"role": "assistant", "type": "message", "content": "24"}
{"role": "assistant", "type": "message", "content": " is"}
{"role": "assistant", "type": "message", "content": " approximately"}
{"role": "assistant", "type": "message", "content": " "}
{"role": "assistant", "type": "message", "content": "1"}
{"role": "assistant", "type": "message", "content": "."}
{"role": "assistant", "type": "message", "content": "42"}
{"role": "assistant", "type": "message", "content": "."}
{"role": "assistant", "type": "message", "end": True}
```
**Note:** Setting `display=True` won't change the behavior of the streaming response, it will just render a display in your terminal.
# Anatomy
Each chunk of the streamed response is a dictionary, that has a "role" key that can be either "assistant" or "computer". The "type" key describes what the chunk is. The "content" key contains the actual content of the chunk.
Every 'message' is made up of chunks, and begins with a "start" chunk, and ends with an "end" chunk. This helps you parse the streamed response into messages.
Let's break down each part of the streamed response.
## Code
In this example, the LLM decided to start writing code first. It could have decided to write a message first, or to only write code, or to only write a message.
Every streamed chunk of type "code" has a format key that specifies the language. In this case it decided to write `python`.
This can be any language defined in [our languages directory.](https://github.com/KillianLucas/open-interpreter/tree/main/interpreter/core/computer/terminal/languages)
```
{"role": "assistant", "type": "code", "format": "python", "start": True}
```
Then, the LLM decided to write some code. The code is sent token-by-token:
```
{"role": "assistant", "type": "code", "format": "python", "content": "34"}
{"role": "assistant", "type": "code", "format": "python", "content": " /"}
{"role": "assistant", "type": "code", "format": "python", "content": " "}
{"role": "assistant", "type": "code", "format": "python", "content": "24"}
```
When the LLM finishes writing code, it will send an "end" chunk:
```
{"role": "assistant", "type": "code", "format": "python", "end": True}
```
## Code Output
After the LLM finishes writing a code block, Open Interpreter will attempt to run it.
**Before** it runs it, the following chunk is sent:
```
{"role": "computer", "type": "confirmation", "format": "execution", "content": {"type": "code", "language": "python", "code": "34 / 24"}}
```
If you check for this object, you can break (or get confirmation) **before** executing the code.
```python
# This example asks the user before running code
for chunk in interpreter.chat("What's 34/24?", stream=True):
if "executing" in chunk:
if input("Press ENTER to run this code.") != "":
break
```
**While** the code is being executed, you'll receive the line of code that's being run:
```
{"role": "computer", "type": "console", "format": "active_line", "content": "1"}
```
We use this to highlight the active line of code on our UI, which keeps the user aware of what Open Interpreter is doing.
You'll then receive its output, if it produces any:
```
{"role": "computer", "type": "console", "format": "output", "content": "1.4166666666666667\n"}
```
When the code is **finished** executing, this flag will be sent:
```
{"role": "computer", "type": "console", "end": True}
```
## Message
Finally, the LLM decided to write a message. This is streamed token-by-token as well:
```
{"role": "assistant", "type": "message", "start": True}
{"role": "assistant", "type": "message", "content": "The"}
{"role": "assistant", "type": "message", "content": " result"}
{"role": "assistant", "type": "message", "content": " of"}
{"role": "assistant", "type": "message", "content": " the"}
{"role": "assistant", "type": "message", "content": " division"}
{"role": "assistant", "type": "message", "content": " "}
{"role": "assistant", "type": "message", "content": "34"}
{"role": "assistant", "type": "message", "content": "/"}
{"role": "assistant", "type": "message", "content": "24"}
{"role": "assistant", "type": "message", "content": " is"}
{"role": "assistant", "type": "message", "content": " approximately"}
{"role": "assistant", "type": "message", "content": " "}
{"role": "assistant", "type": "message", "content": "1"}
{"role": "assistant", "type": "message", "content": "."}
{"role": "assistant", "type": "message", "content": "42"}
{"role": "assistant", "type": "message", "content": "."}
{"role": "assistant", "type": "message", "end": True}
```
For an example in JavaScript on how you might process these streamed chunks, see the [migration guide](https://github.com/KillianLucas/open-interpreter/blob/main/docs/NCU_MIGRATION_GUIDE.md)
|