Polling Transport

Use polling instead of SSE for environments that can't maintain long-lived HTTP connections.


When to Use Polling

Some environments cannot hold open Server-Sent Events (SSE) connections:

  • Convex actions — HTTP requests must complete within a timeout
  • Cloudflare Workers — no long-lived inbound connections
  • Restricted edge runtimes — firewalls or proxies that drop idle connections
  • Serverless functions — Lambda, Cloud Functions with short timeouts

In these cases, set stream: false to switch from SSE to lightweight polling. The SDK automatically polls a status endpoint and only fetches full data when the status changes.


Client-Level Default

Set polling as the default transport for all operations:

typescript
1import { inference } from '@inferencesh/sdk';23const client = inference({4  apiKey: 'inf_your_key',5  stream: false,          // Use polling instead of SSE6  pollIntervalMs: 2000,   // Poll every 2 seconds (default)7});

All client.run(), agent.sendMessage(), and AgentChatProvider calls will use polling unless overridden.


Task Polling

Override the transport per-call with stream: false on run():

typescript
1const result = await client.run(2  { app: 'infsh/flux', input: { prompt: 'A sunset' } },3  {4    stream: false,5    pollIntervalMs: 3000,  // Optional: override interval6    onUpdate: (task) => {7      console.log('Status:', task.status);8    },9  }10);1112console.log('Output:', result.output);

How it works:

  1. Task is created via POST /apps/run (same as streaming)
  2. SDK polls GET /tasks/{id}/status every pollIntervalMs
  3. When the status changes, it fetches the full task via GET /tasks/{id}
  4. onUpdate fires with the full task data
  5. Promise resolves on completion, rejects with TaskFailedError on failure (its message is the task's error when present) or TaskCancelledError on cancellation. Both carry the task as it ended on task; a cancelled task's task.output is what its app returned before stopping, or null

When status polling cannot recover (for example repeated network errors), set maxReconnects on run() options (default 5) to control retry behavior.


Agent Polling

Imperative SDK

typescript
1const client = inference({ apiKey: 'inf_your_key' });23const agent = client.agent('my-team/support-agent@latest');45const response = await agent.sendMessage('Hello!', {6  stream: false,7  pollIntervalMs: 2000,8  onMessage: (message) => {9    console.log('Message:', message.content);10  },11  onChat: (chat) => {12    console.log('Chat status:', chat.status);13  },14  onToolCall: (invocation) => {15    console.log('Tool call:', invocation.name);16  },17});

How it works:

  1. Message is sent via POST /agents/run (same as streaming)
  2. SDK polls GET /chats/{id}/status every pollIntervalMs
  3. When the status changes, it fetches chat metadata via GET /chats/{id} and dispatches onChat
  4. New and updated messages arrive from chat_messages stream events (streaming mode) or require a separate list-messages call (POST /chats/{id}/messages/list from @inferencesh/sdk v0.38.0, GET /chats/{id}/messages before) — GET /chats/{id} does not include message history
  5. Client tool invocations (in_progress, or awaiting_input for backwards compatibility) are dispatched to onToolCall
  6. Promise resolves when the chat becomes idle

Transient status poll failures (for example a single network error on GET /chats/{id}/status) log a [Agent] Poll error: warning and polling continues. @inferencesh/sdk ≥ v0.28.0 rejects sendMessage with the last poll error once polling gives up after repeated failures; earlier releases could wait forever. Use onError on AgentChatProvider to surface persistent transport failures to your UI.

React Provider

tsx
1import { AgentChatProvider, useAgentChat } from '@inferencesh/sdk/agent';23function App() {4  return (5    <AgentChatProvider6      client={client}7      agentConfig={{ agent: 'my-team/support-agent@latest' }}8      stream={false}9      pollIntervalMs={2000}10    >11      <ChatUI />12    </AgentChatProvider>13  );14}

How it works (React provider):

  1. First message: POST /chats (create chat with agent ref), then POST /chats/{id}/messages
  2. Follow-ups: POST /chats/{id}/messages only
  3. SDK polls GET /chats/{id}/status every pollIntervalMs
  4. When the status changes, the SDK fetches chat metadata (GET /chats/{id}) and message history (POST /chats/{id}/messages/list from v0.38.0, GET /chats/{id}/messages before) and merges them into chat state — GET /chats/{id} alone does not include messages
  5. New and updated messages are available through useAgentChat() (chat.chat_messages)
  6. sendMessage resolves when the POST completes; assistant replies arrive via polling or SSE

@inferencesh/sdk ≥ v0.33.0: with polling, the provider applies the user message from the POST response right away. Status polling refetches messages only when chat.status changes, so without that a follow-up sent while the chat is busy would stay hidden until the turn ends. With SSE, the stream owns the message instead (see Agent chat streams).

The provider uses the same status-polling mechanism as the imperative SDK after the message is sent. All hooks (useAgentChat, useAgentActions) work identically regardless of transport.

Loading older messages

Long chats load only the 50 newest messages on connect. v0.6.41+: use useAgentActions().loadOlderMessages() to fetch older pages — the provider tracks the cursor from the initial load, prepends deduplicated messages to useAgentChat().messages, and returns true when more pages exist. Check hasOlderMessages before showing a scroll-up affordance. Works the same with SSE and polling.

tsx
1const { messages } = useAgentChat();2const { loadOlderMessages, hasOlderMessages } = useAgentActions();34// Call when the user scrolls near the top of the message list5if (hasOlderMessages) {6  await loadOlderMessages();7}

Optional lifecycle callbacks (available on AgentChatProvider with both SSE and polling):

PropWhen it fires
onChatCreatedA new chat ID is assigned after the first message
onStatusChangeConnection status changes (connecting, streaming, idle). v0.23.0+ reports connecting while the chat stream waits to retry after a network drop. It is never called with error: when reconnecting gives up, onError fires and useAgentChat().connectionStatus becomes 'error'
onTurnEndThe agent's turn ends — chat status transitions from busy to idle, completed, or another non-busy state. Receives the full ChatDTO. Use to refetch host data after agent tool actions without polling or window-focus workarounds.
onErrorStream or poll transport errors, failed sends, or tool submission failures
tsx
1<AgentChatProvider2  client={client}3  agentConfig={{ agent: 'my-team/support-agent@latest' }}4  stream={false}5  onError={(error) => console.error('Agent error:', error)}6  onStatusChange={(status) => console.log('Status:', status)}7  onTurnEnd={(chat) => refetchDashboard(chat.id)}8>9  <ChatUI />10</AgentChatProvider>

onError on the provider is separate from the client-level onError in inference({ onError }), which handles REST, HttpClient.fetch(), and failed initial stream connections (SSE and NDJSON). See Error Handling.


Configuration Reference

OptionTypeDefaultScope
streambooleantrueClient, run(), sendMessage(), Provider
pollIntervalMsnumber2000Client, run(), sendMessage(), Provider
maxReconnectsnumber5run() when stream: false

Precedence: Per-call options override provider props, which override client defaults.


Comparison: Streaming vs Polling

SSE (default)Polling
LatencyReal-timeUp to pollIntervalMs delay
Partial updatesYes (onPartialUpdate)No (full fetch only)
ConnectionLong-lived HTTPShort-lived requests
Edge compatibleNoYes
BandwidthLower (deltas only)Higher (status checks + full fetches)

Use SSE when you can. Use polling when you must.

we use cookies

we use cookies to ensure you get the best experience on our website. for more information on how we use cookies, please see our cookie policy.

by clicking "accept", you agree to our use of cookies.
learn more.