Streaming responses
General considerations for progressively receiving and displaying a generated response.
Working scaffold: The transport, event names, and completion markers below are not defined for ONE. Confirm the supported streaming protocol before adding executable examples.
When streaming helps
Streaming can make a long-running response feel more responsive by delivering parts as they become available. It also changes how clients parse, render, cancel, and recover from a request.
Client responsibilities
- Enable streaming using the documented request option.
- Read the documented event or chunk format incrementally.
- Append only the expected content to the user-visible response.
- Detect completion, interruption, and error events.
- Release the connection and any reader resources when finished or cancelled.
The exact transport may be server-sent events, a streamed HTTP body, or another mechanism. Do not assume that chunks are complete words, JSON objects, or application messages.
Production considerations
Handle disconnects and cancellation; avoid buffering an unbounded response; and ensure that final usage or metadata is processed when the protocol provides it. Keep logs free of sensitive prompts and credentials.