Agent Streaming

Handle real-time message updates and tool calls.

Both SDKs read one server-sent event stream per chat (/chats/{id}/stream). It carries three kinds of updates: chat_messages (a message as stored, whole, every time it changes), chats and agent_runs (run state), and delta (token batches for the message being generated). onMessage always gives you the complete message so far, so there is nothing to concatenate; onDelta gives you the increments. Messages with role: "event" (for example hook handler runs) arrive on the same channel but are not LLM turns — filter or render them separately if you only care about user/assistant/tool content.

Basic Streaming

Pass callbacks to sendMessage and it waits until the agent's turn is over, delivering updates as they arrive.

1# Tokens as they are generated. d.delta is this batch,2# d.output everything received so far for d.message_id.3def on_delta(d):4    print(d.delta.get("response", ""), end="", flush=True)56# Each message once it is stored (user, assistant, tool, event, …).7def on_message(message):8    if message.get("role") == "assistant" and message.get("status") == "ready":9        print(f"\n[done] {message['id']}")1011agent.send_message("Tell me a story", on_delta=on_delta, on_message=on_message)

onDelta is streaming-only. With stream: false the SDK polls the chat instead, there are no deltas, and each message arrives whole through onMessage.

Tool Call Callbacks

A client tool (one the agent declares but your code executes) reaches you through onToolCall with { id, name, args }. Execute it and submit the result; the agent continues from there.

1def on_tool_call(tool_call):2    print(f"[tool] {tool_call.name} {tool_call.args}")3    result = my_tools[tool_call.name](**tool_call.args)4    agent.submit_tool_result(tool_call.id, str(result))56agent.send_message("What's 42 * 17?", on_tool_call=on_tool_call)

Each invocation is dispatched once. A widget interaction submits an action instead of a string: submitToolResult(id, { action: { type: 'confirm' }, form_data: { ... } }).

Without Callbacks

With no callbacks, sendMessage returns as soon as the server has accepted the message; the agent keeps working in the background. Use this when another consumer renders the chat, or read the result later:

1agent.send_message("Summarise this thread")23# Iterate the stream yourself; it ends when the chat goes idle.4for message in agent.stream_messages():5    print(message.get("status"))67# Or run to completion and take the structured output8output = agent.run("Extract the action items")

A turn keeps running on the server when the client disconnects. A dropped sendMessage or send_message stream does not stop the agent; attach again with startStreaming() or stream_messages(), or poll the chat's messages. See Reconnection.

run() always polls and returns chat.output, the result of the agent's finish tool, or null when it finished without calling it.

Async

1from inferencesh import AsyncInference2import asyncio34async def main():5    client = AsyncInference(api_key="inf_...")6    agent = client.agent("my-org/assistant")78    await agent.send_message("Hello")9    async for message in agent.stream_messages():10        print(message.get("status"))1112asyncio.run(main())

Send Options

1agent.send_message(2    "Hello",3    files=None,                   # bytes or data-URI strings, uploaded first4    on_delta=on_delta,            # AgentDelta(message_id, delta, output, seq)5    on_message=on_message,        # ChatMessageDTO, whole message each update6    on_tool_call=on_tool_call,    # ToolCallInfo(id, name, args)7)

A turn that stops on a tool approval or an authorization the agent needs stays open: the JavaScript SDK keeps waiting for it to be resolved, from your onToolCall handler or from the inference.sh app. A client that cannot resolve it passes a signal and aborts; sendMessage then rejects with signal.reason while the agent stays parked server-side. The Python SDK returns from send_message at that point instead.