Inference Logoinference.sh

Streaming

Handle real-time message updates and tool calls.

Note: Agent streaming sends the full message content on each update, not incremental chunks. You don't need to concatenate—each update gives you the complete message so far.


Basic Streaming

typescript
1const agent = client.agent({...});23for await (const message of agent.sendMessage('Tell me a story', { stream: true })) {4  console.log(message.text);  // Full message content so far5}

onMessage Callback

typescript
1await agent.sendMessage('Tell me a story', {2  onMessage: (text, done) => {3    if (done) {4      console.log(`\n[Complete] Final: ${text}`);5    } else {6      console.log(`Current message: ${text}`);7    }8  }9});

onToolCall Callback

typescript
1await agent.sendMessage("What's 42 * 17?", {2  onToolCall: (toolCall) => {3    console.log(`Calling: ${toolCall.name}`);4    console.log(`Args: ${JSON.stringify(toolCall.arguments)}`);5  }6});

Combined Callbacks

typescript
1await agent.sendMessage('Generate an image of a sunset', {2  onMessage: (text, done) => {3    if (done) {4      console.log(`\nFinal: ${text}`);5    } else {6      console.log(`Progress: ${text.slice(0, 50)}...`);7    }8  },9  onToolCall: (toolCall) => {10    console.log(`\n[Tool: ${toolCall.name}]`);11  }12});

Manual Tool Handling

For tools without handlers:

typescript
1const calculator = tool('calculator')2  .describe('Do math')3  .param('expression', string('Expression'))4  .build();  // No handler56const agent = client.agent({7  core_app: { ref: 'infsh/claude-sonnet-4@latest' },8  tools: [calculator]9});1011const response = await agent.sendMessage("What's 42 * 17?");1213if (response.pendingToolCalls) {14  for (const tc of response.pendingToolCalls) {15    const result = eval(tc.arguments.expression);16    await agent.submitToolResult(tc.id, String(result));17  }18}

Send Options

typescript
1interface SendMessageOptions {2  stream?: boolean;3  onMessage?: (text: string, done: boolean) => void;4  onToolCall?: (toolCall: ToolCallInfo) => void;5}

we use cookies

we use cookies to ensure you get the best experience on our website. for more information on how we use cookies, please see our cookie policy.

by clicking "accept", you agree to our use of cookies.
learn more.