Streaming

Real-time updates via Server-Sent Events (SSE).

Send X-API-Version: 2 on the initial request (same as other REST endpoints). SSE event payloads are not wrapped in the version 1 envelope. See REST overview — API version.

Stream Task Updates

GET /tasks/:id/stream

Stream task status updates in real-time.

Headers

1Accept: text/event-stream

Event Format

1event: update2data: {"id":"task_abc","status":7,"logs":"Processing..."}34event: update5data: {"id":"task_abc","status":10,"output":{...}}67event: done8data: {}

Event Types

cURL Example

1curl -N https://api.inference.sh/tasks/task_abc123/stream \2  -H "Authorization: Bearer inf_your_key" \3  -H "X-API-Version: 2" \4  -H "Accept: text/event-stream"

Stream workspace events

GET /events

Real-time updates for resources in your workspace. The inference shell app uses this stream to refresh user profile, billing transactions, and engine state without polling.

Requires authentication with teams:read scope.

Format negotiation

Send X-API-Version: 2 on the initial request.

NDJSON messages

Each line is a JSON object. Heartbeats use {"type":"heartbeat"} (every 10 seconds).

Typed updates include an event field set to the resource table name:

1{"event":"engines","data":{"id":"eng_abc","status":"running",...}}2{"event":"transactions","data":{"id":"txn_xyz","amount":500,...}}3{"event":"users","data":{"id":"usr_123","balance":10000,...}}

Subscribe to the event names you care about (engines, transactions, users). Partial field updates may include a fields array alongside data. The id field is always included in partial payloads so clients can correlate updates with existing records, even when fields omits it.

SSE format

1event: engines2data: {"id":"eng_abc","status":"running",...}34: heartbeat

The event name matches the resource type (same table names as NDJSON).

cURL example

1curl -N https://api.inference.sh/events \2  -H "Authorization: Bearer inf_your_key" \3  -H "X-API-Version: 2" \4  -H "Accept: application/x-ndjson"

When to use this vs other streams

There is no last_event_id resume on /events — open a new connection after disconnects.

Stream engine updates

GET /engines/{id}/stream

Stream status changes for a single private engine. Requires engines:read scope.

Send Accept: text/event-stream and X-API-Version: 2. Events use the same SSE pattern as task streaming: update events with engine DTO payloads, done when the stream closes, and heartbeat comments every 10 seconds.

1curl -N https://api.inference.sh/engines/eng_abc123/stream \2  -H "Authorization: Bearer inf_your_key" \3  -H "X-API-Version: 2" \4  -H "Accept: text/event-stream"

In the JavaScript SDK: client.engines.stream(engineId).

Stream chat updates

GET /chats/:id/stream

Unified real-time stream for agent chat state, messages, and runs. Used by belt chat watch, the JavaScript SDK (client.chats.stream(chatId)), and client.agent().sendMessage() when streaming.

Requires conversations:read scope. Send X-API-Version: 2 and Accept: text/event-stream (default) or application/x-ndjson.

Initial snapshot

On connect (including reconnect), the server sends one chats event with the current chat status (busy, idle, awaiting_input, …) derived from the active run. Message history is omitted from the snapshot to keep payloads small — call List chat messages to load the transcript, including queued messages.

Event types

Tool invocation progress arrives as chat_messages updates — the server republishes the parent message when an invocation status changes.

Queued messages emit chat_messages with status: "queued", then status: "ready" when the agent consumes them on its next turn. When a run completes and a new run starts for queued messages, you receive agent_runs events for both.

Partial updates

Some chat_messages events are partial updates — for example when status changes from queued to ready. These use a {data, fields} envelope (same pattern as task partial updates):

1{"event":"chat_messages","data":{"id":"msg_1","chat_id":"chat_abc","status":"ready"},"fields":["id","status","chat_id"]}

The id field is always present in data, even when fields omits it. Merge partial updates into your local message state instead of replacing the full object. The JavaScript SDK handles this via onPartialData on chat streams; Python iterators unwrap the envelope automatically.

Full-object chat_messages events (message created, content streaming) omit the fields wrapper.

Heartbeats are sent every 10 seconds (: heartbeat in SSE, {"type":"heartbeat"} in NDJSON).

SSE example

1event: chats2data: {"id":"chat_abc","status":"busy",...}34event: chat_messages5data: {"id":"msg_1","role":"user","status":"queued",...}67: heartbeat89event: chat_messages10data: {"data":{"id":"msg_1","chat_id":"chat_abc","status":"ready"},"fields":["id","status","chat_id"]}1112event: agent_runs13data: {"id":"run_1","state":"completed",...}

cURL example

1curl -N https://api.inference.sh/chats/chat_abc123/stream \2  -H "Authorization: Bearer inf_your_key" \3  -H "X-API-Version: 2" \4  -H "Accept: application/x-ndjson"

Reconnection

Open a new connection to /chats/:id/stream. The initial snapshot reflects the current chat state — not the state from when you last disconnected. Fetch List chat messages for any messages you missed while offline.

Unlike task streams, chat streams stay open across multiple agent turns. They do not emit a done event when a single turn completes. Fetch structured results (if any) with GET /chats/:id — see Get Chat.

Related endpoints

Connection Parameters

Handling Reconnection

SSE supports automatic reconnection. Use the Last-Event-ID header:

1curl -N https://api.inference.sh/tasks/task_abc123/stream \2  -H "Authorization: Bearer inf_your_key" \3  -H "X-API-Version: 2" \4  -H "Accept: text/event-stream" \5  -H "Last-Event-ID: evt_xyz"

JavaScript Example

1const eventSource = new EventSource(2  'https://api.inference.sh/tasks/task_abc123/stream',3  {4    headers: {5      'Authorization': 'Bearer inf_your_key'6    }7  }8);910eventSource.addEventListener('update', (event) => {11  const data = JSON.parse(event.data);12  console.log('Status:', data.status);13});1415eventSource.addEventListener('done', () => {16  eventSource.close();17});