Streaming API

SSE Format

When stream: true is set in the request, the response is delivered as a stream of Server-Sent Events. The connection uses the following HTTP headers:

Wire Format

Each event is a line prefixed with data: followed by a JSON object and terminated by two newlines:

The stream ends with a [DONE] sentinel:

Clients should parse each data: line, check for [DONE], and decode the JSON for all other lines.

Chunk Structure

Each streaming chunk follows this schema:

Delta Object

The delta field in each choice contains the incremental content:

finish_reason Values

Per-Chunk Log Probabilities

When logprobs: true is set in the request, each streaming choice includes a logprobs object with per-token log probabilities for the tokens in that chunk. The structure mirrors the non-streaming logprobs format:

Each entry in the content and refusal arrays contains token, logprob, bytes, and top_logprobs fields, identical to the non-streaming logprobs format. When logprobs is not requested, the field is null in all chunks.

Refusal Streaming

When the model refuses a request due to content policy or safety filters, the refusal text is delivered incrementally via delta.refusal instead of delta.content. The finish_reason is typically "stop" and content will be null in the final response.

Clients should concatenate delta.refusal strings across chunks just like delta.content. When logprobs: true is set, refusal token probabilities appear in logprobs.refusal on each chunk.

Annotated Stream Example

Usage Reporting

To receive token usage data in a streaming response, set stream_options in your request:

When enabled, the final chunk (the one with finish_reason set) includes a usage object:

The prompt_tokens_details.cached_tokens field shows how many prompt tokens were served from the prefix cache, reducing time-to-first-token.

The completion_tokens_details object provides a breakdown of output tokens. For reasoning models (when reasoning_effort is set), reasoning_tokens shows how many tokens were used for internal reasoning. When prediction is used, accepted_prediction_tokens and rejected_prediction_tokens indicate how effective the prediction was.

Without stream_options.include_usage, the usage field is null in all chunks. Token usage is still tracked internally for billing.

Tool Call Streaming

When the model generates tool calls, the delta.tool_calls field contains incremental data. Tool calls are streamed across multiple chunks:

• The first chunk for a tool call includes id, type, and the function name.
• Subsequent chunks include only the arguments string, accumulated incrementally.
• The index field identifies which tool call the delta belongs to (for parallel tool calls).

Tool Call Stream Example

Clients should concatenate the arguments strings across chunks and parse the complete JSON after finish_reason: "tool_calls".

Vendor Events

The platform emits vendor-prefixed events (x_*) as inline data: lines within the Chat Completions stream. These events are not standard OpenAI SSE events; they carry a JSON object with a type field that starts with x_. Standard OpenAI SDK clients silently discard unrecognized data lines, so these events are backward compatible.

Vendor events are emitted by the platform when platform-specific features are active (research mode, deep think, analyst mode, artifacts, and interactive user prompts). They flow inline between regular chat.completion.chunk data lines. Custom consumers should inspect the type field of each data line and handle both OpenAI-standard chunk objects and x_* vendor payloads.

Wire Format

Research Events (x_research.*)

Emitted during agentic research loops (research mode).

Deep Think Events (x_deep_think.*)

Emitted during deep think (multi-step research with planning and synthesis).

Artifact Events (x_artifact.*)

Emitted when code artifacts are created or updated during generation.

Mockup Events (x_mockup.*)

Emitted when an agentic-mode response creates or updates a multi-file mockup bundle (see create_mockup and x_update_mockup). Bundle files are reachable from the preview iframe at GET /v1/mockups/{bundleId}/{path}.

Ask User Events (x_ask_user.*)

Emitted when the model needs clarification before it can continue.

Context Fork Event (x_context_fork)

Emitted when the user's message triggered creation of a new conversation branch. Fields: branch_id, branch_name, message_count.

Chat Metadata Event (x_chat.metadata)

Emitted as the final data event before [DONE] on dashboard chat streams. Contains server-side persistence identifiers, context budget breakdown, and combined token usage (including any research or deep think overhead).

Analyst Events (x_analyst.*)

Emitted when analyst mode builds or refreshes the workspace context brief.

Error Handling

Pre-Stream Errors

Errors that occur before the SSE connection is established return standard HTTP status codes (400, 401, 404, 429, 503). The response body is a JSON error object, not an SSE stream.

Mid-Stream Errors

Errors that occur during an active stream use the event: error SSE event type (not the standard data: prefix). The [DONE] sentinel is always sent after the error:

Error Types

Client Disconnect

When a client disconnects during a stream, the router detects the broken connection and sends a cancellation request to the backend agent. The agent stops generation to free resources. Any tokens generated before disconnection are still billed.

Heartbeats

During idle periods (no chunks for 15 seconds), the router sends SSE comment frames to keep the connection alive:

Per the SSE specification, lines beginning with a colon are comments that clients silently ignore. These heartbeats prevent reverse proxies and load balancers from closing idle connections due to read timeouts. Heartbeats do not reset the idle stream timeout tracker.

Client Examples

The --no-buffer flag disables curl's output buffering so chunks are displayed as they arrive.