Skip to content

Commit bc9a1ca

Browse files
docs(readme): assistant streaming (#719)
1 parent bb4bce3 commit bc9a1ca

File tree

1 file changed

+145
-13
lines changed

1 file changed

+145
-13
lines changed

helpers.md

+145-13
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,140 @@
1-
# Chat Completion Helpers
1+
# Streaming Helpers
22

3-
## Streaming Responses
3+
OpenAI supports streaming responses when interacting with the [Chat](#chat-streaming) or [Assistant](#assistant-streaming-api) APIs.
4+
5+
## Assistant Streaming API
6+
7+
OpenAI supports streaming responses from Assistants. The SDK provides convenience wrappers around the API
8+
so you can subscribe to the types of events you are interested in as well as receive accumulated responses.
9+
10+
More information can be found in the documentation: [Assistant Streaming](https://platform.openai.com/docs/assistants/overview?lang=node.js)
11+
12+
#### An example of creating a run and subscribing to some events
13+
14+
```ts
15+
const run = openai.beta.threads.runs
16+
.createAndStream(thread.id, {
17+
assistant_id: assistant.id,
18+
})
19+
.on('textCreated', (text) => process.stdout.write('\nassistant > '))
20+
.on('textDelta', (textDelta, snapshot) => process.stdout.write(textDelta.value))
21+
.on('toolCallCreated', (toolCall) => process.stdout.write(`\nassistant > ${toolCall.type}\n\n`))
22+
.on('toolCallDelta', (toolCallDelta, snapshot) => {
23+
if (toolCallDelta.type === 'code_interpreter') {
24+
if (toolCallDelta.code_interpreter.input) {
25+
process.stdout.write(toolCallDelta.code_interpreter.input);
26+
}
27+
if (toolCallDelta.code_interpreter.outputs) {
28+
process.stdout.write('\noutput >\n');
29+
toolCallDelta.code_interpreter.outputs.forEach((output) => {
30+
if (output.type === 'logs') {
31+
process.stdout.write(`\n${output.logs}\n`);
32+
}
33+
});
34+
}
35+
}
36+
});
37+
```
38+
39+
### Assistant Events
40+
41+
The assistant API provides events you can subscribe to for the following events.
42+
43+
```ts
44+
.on('event', (event: AssistantStreamEvent) => ...)
45+
```
46+
47+
This allows you to subscribe to all the possible raw events sent by the OpenAI streaming API.
48+
In many cases it will be more convenient to subscribe to a more specific set of events for your use case.
49+
50+
More information on the types of events can be found here: [Events](https://platform.openai.com/docs/api-reference/assistants-streaming/events)
51+
52+
```ts
53+
.on('runStepCreated', (runStep: RunStep) => ...)
54+
.on('runStepDelta', (delta: RunStepDelta, snapshot: RunStep) => ...)
55+
.on('runStepDone', (runStep: RunStep) => ...)
56+
```
57+
58+
These events allow you to subscribe to the creation, delta and completion of a RunStep.
59+
60+
For more information on how Runs and RunSteps work see the documentation [Runs and RunSteps](https://platform.openai.com/docs/assistants/how-it-works/runs-and-run-steps)
61+
62+
```ts
63+
.on('messageCreated', (message: Message) => ...)
64+
.on('messageDelta', (delta: MessageDelta, snapshot: Message) => ...)
65+
.on('messageDone', (message: Message) => ...)
66+
```
67+
68+
This allows you to subscribe to Message creation, delta and completion events. Messages can contain
69+
different types of content that can be sent from a model (and events are available for specific content types).
70+
For convenience, the delta event includes both the incremental update and an accumulated snapshot of the content.
71+
72+
More information on messages can be found
73+
on in the documentation page [Message](https://platform.openai.com/docs/api-reference/messages/object).
74+
75+
```ts
76+
.on('textCreated', (content: Text) => ...)
77+
.on('textDelta', (delta: RunStepDelta, snapshot: Text) => ...)
78+
.on('textDone', (content: Text, snapshot: Message) => ...)
79+
```
80+
81+
These events allow you to subscribe to the creation, delta and completion of a Text content (a specific type of message).
82+
For convenience, the delta event includes both the incremental update and an accumulated snapshot of the content.
83+
84+
```ts
85+
.on('imageFileDone', (content: ImageFile, snapshot: Message) => ...)
86+
```
87+
88+
Image files are not sent incrementally so an event is provided for when a image file is available.
89+
90+
```ts
91+
.on('toolCallCreated', (toolCall: ToolCall) => ...)
92+
.on('toolCallDelta', (delta: RunStepDelta, snapshot: ToolCall) => ...)
93+
.on('toolCallDone', (toolCall: ToolCall) => ...)
94+
```
95+
96+
These events allow you to subscribe to events for the creation, delta and completion of a ToolCall.
97+
98+
More information on tools can be found here [Tools](https://platform.openai.com/docs/assistants/tools)
99+
100+
```ts
101+
.on('end', () => ...)
102+
```
103+
104+
The last event send when a stream ends.
105+
106+
### Assistant Methods
107+
108+
The assistant streaming object also provides a few methods for convenience:
109+
110+
```ts
111+
.currentEvent()
112+
113+
.currentRun()
114+
115+
.currentMessageSnapshot()
116+
117+
.currentRunStepSnapshot()
118+
```
119+
120+
These methods are provided to allow you to access additional context from within event handlers. In many cases
121+
the handlers should include all the information you need for processing, but if additional context is required it
122+
can be accessed.
123+
124+
Note: There is not always a relevant context in certain situations (these will be undefined in those cases).
125+
126+
```ts
127+
await.finalMessages();
128+
129+
await.finalRunSteps();
130+
```
131+
132+
These methods are provided for convenience to collect information at the end of a stream. Calling these events
133+
will trigger consumption of the stream until completion and then return the relevant accumulated objects.
134+
135+
## Chat Streaming
136+
137+
### Streaming Responses
4138

5139
```ts
6140
openai.chat.completions.stream({ stream?: false, … }, options?): ChatCompletionStreamingRunner
@@ -18,7 +152,7 @@ If you need to cancel a stream, you can `break` from a `for await` loop or call
18152
19153
See an example of streaming helpers in action in [`examples/stream.ts`](examples/stream.ts).
20154
21-
## Automated Function Calls
155+
### Automated Function Calls
22156
23157
```ts
24158
openai.chat.completions.runTools({ stream: false, … }, options?): ChatCompletionRunner
@@ -69,9 +203,7 @@ See an example of automated function calls in action in
69203

70204
Note, `runFunctions` was also previously available, but has been deprecated in favor of `runTools`.
71205

72-
## Runner API
73-
74-
### Events
206+
### Chat Events
75207

76208
#### `.on('connect', () => …)`
77209

@@ -148,7 +280,7 @@ The event fired at the end, returning the total usage of the call.
148280

149281
The last event fired in the stream.
150282

151-
### Methods
283+
### Chat Methods
152284

153285
#### `.abort()`
154286

@@ -190,7 +322,7 @@ A promise which resolves with the last message with a `role: "function"`. Throws
190322

191323
A promise which resolves with the total usage.
192324

193-
### Fields
325+
### Chat Fields
194326

195327
#### `.messages`
196328

@@ -200,9 +332,9 @@ A mutable array of all messages in the conversation.
200332

201333
The underlying `AbortController` for the runner.
202334

203-
## Examples
335+
### Chat Examples
204336

205-
### Abort on a function call
337+
#### Abort on a function call
206338

207339
If you have a function call flow which you intend to _end_ with a certain function call, then you can use the second
208340
argument `runner` given to the function to either mutate `runner.messages` or call `runner.abort()`.
@@ -238,7 +370,7 @@ async function main() {
238370
main();
239371
```
240372
241-
### Integrate with `zod`
373+
#### Integrate with `zod`
242374
243375
[`zod`](https://www.npmjs.com/package/zod) is a schema validation library which can help with validating the
244376
assistant's response to make sure it conforms to a schema. Paired with [`zod-to-json-schema`](https://www.npmjs.com/package/zod-to-json-schema), the validation schema also acts as the `parameters` JSON Schema passed to the API.
@@ -287,10 +419,10 @@ main();
287419
288420
See a more fully-fledged example in [`examples/function-call-helpers-zod.ts`](examples/function-call-helpers-zod.ts).
289421

290-
### Integrate with Next.JS
422+
#### Integrate with Next.JS
291423

292424
See an example of a Next.JS integration here [`examples/stream-to-client-next.ts`](examples/stream-to-client-next.ts).
293425

294-
### Proxy Streaming to a Browser
426+
#### Proxy Streaming to a Browser
295427

296428
See an example of using express to stream to a browser here [`examples/stream-to-client-express.ts`](examples/stream-to-client-express.ts).

0 commit comments

Comments
 (0)