Python SDK for inference shell agents and apps.
The Python SDK provides both sync and async clients for running apps and chatting with agents.
Installation
1pip install inferenceshFor async support:
1pip install inferencesh[async]Client API
The inference() and async_inference() factories return sync Inference and async AsyncInference clients. Responses are unwrapped from the { "data", "messages" } envelope and errors are RFC 9457 problem details.
| Property | Purpose |
|---|---|
client.tasks | Run, get, cancel, delete, wait, and stream tasks; files and delete_files for a task's input and output files |
client.files | Upload files |
client.agents | Create agents (agents.create() is the same as agent() without per-chat context) |
client.sessions | List, inspect, and end sessions; use client.session() for a context manager |
client.live | Run a stream function and dial its socket (async only; requires inferencesh[async]) |
client.sockets | HTTP helpers for stream-task sockets; open() redials with a fresh credential (async only) |
api_key is optional (inferencesh ≥ v0.22.0). Omit it and the client reads the INFERENCE_API_KEY environment variable; an explicit api_key wins. With neither, the constructor raises ValueError. The same applies to Inference() and AsyncInference().
1from inferencesh import inference, async_inference23client = inference() # reads INFERENCE_API_KEY4client = inference(api_key="inf_your_key") # explicit key wins5aclient = async_inference() # same rule for the async clientclient.run() and client.upload_file() remain supported as legacy aliases for client.tasks.run() and client.files.upload().
Prefer client.tasks.run() for new code.
The JavaScript SDK exposes additional namespaces (apps, chats, engines, search, and more). For features not listed above, use the REST API or see the SDK overview — client structure parity table.
Reference strings (Ref)
CLI commands, task inputs, and agent tools accept refs in [type/]namespace/name[@version][:function] form. The same grammar is used across the platform (Go apitypes.Ref.Parse is the source of truth). Parse and format refs in Python with Ref from inferencesh (inferencesh ≥ v0.7.27):
1from inferencesh import Ref23ref = Ref.parse("app/acme/voice-loop@v3:stream")4assert ref.type == "app"5assert ref.namespace == "acme"6assert ref.name == "voice-loop"7assert ref.version_id == "v3"8assert ref.function == "stream"9assert ref.is_typed() is True10assert str(ref) == "app/acme/voice-loop@v3:stream"1112plain = Ref.parse("acme/widget@latest")13assert plain.version_id == "" # @latest is treated as unversioned1415ref, has_namespace = Ref.try_parse("acme/widget")16assert has_namespace is True| Piece | Meaning |
|---|---|
type/ | Optional resource kind: app, agent, skill, or knowledge |
namespace/name | Workspace slug and resource slug (acme/widget) |
@version | Version pin (abc123, v2, …). @latest is ignored (unversioned) |
:function | Function name on multi-function apps (only after @version when both are present) |
Unknown first path segments are not treated as a type prefix. For example, not-a-type/acme/widget is a single opaque name. Colons inside the name before @ stay in the name (acme/weird:name@ver → name weird:name, version ver).
Helpers: full_name() → namespace/name, qualified_name() → type/namespace/name when typed, otherwise namespace/name.
Live apps and stream sockets
Stream functions keep a duplex socket open for the life of the task. Use the async client (pip install inferencesh[async]):
1from inferencesh import async_inference23client = async_inference(api_key="inf_...")45task, session = await client.live(6 {"app": "xai/grok-voice", "function": "talk", "input": {"voice": "eve"}},7 on_clear=lambda field: player.flush(field), # app sent {"$clear": field}8)910await session.send(mic_frame) # bytes for binary live fields11async for frame in session:12 handle(frame)13await session.close()1415# After a reload, redial with a fresh credential (task id or run response)16session = await client.sockets.open(task["id"], on_clear=lambda f: player.flush(f))The sync client.sockets namespace supports get, list, for_task, access, and delete over HTTP. Calling open() on the sync client raises NotImplementedError — use AsyncInference instead.
When the app cuts a live output short it may send a $clear control frame. Pass on_clear to client.live() or client.sockets.open() (or AsyncLiveSession callbacks) to drop buffered audio or UI for that field; without on_clear, the frame arrives as an ordinary JSON patch.
→ Streaming apps — app-side Live, PCM fields, and local testing with belt app test --stream
Running Apps
Basic Usage
1from inferencesh import inference23client = inference(api_key="inf_your_key")45# Run and wait for completion (default)6result = client.tasks.run({7 "app": "infsh/echo",8 "input": {"message": "Hello from Python!"}9})1011print(result["output"])Streaming Progress
1for update in client.tasks.run({"app": "infsh/flux-schnell", "input": {...}}, stream=True):2 print(f"Status: {update['status']}")3 if update.get("logs"):4 print(update["logs"][-1])Partial {data, fields} envelopes are unwrapped automatically. If the stream ends without a terminal event, the SDK reconciles via GET /tasks/:id and populates stream.result. See Streaming — partial updates and reconciliation.
Fire and Forget
1task = client.tasks.run({"app": "infsh/slow-task", "input": {...}}, wait=False)2print(f"Task ID: {task['id']}")Task Management
1from inferencesh import TaskStatus23# Get current task state4task = client.tasks.get(task_id)56# Cancel a running task7client.tasks.cancel(task_id)89# Wait for a task started with wait=False (optional timeout in seconds)10result = client.tasks.wait_for_completion(task_id, timeout=300)1112# Stream updates for an existing task13with client.tasks.stream(task_id) as stream:14 for update in stream:15 if update["status"] == TaskStatus.COMPLETED:16 print(update["output"])17 breakTask files and deletion
inferencesh ≥ v0.19.0 adds delete(), files(), and delete_files() on client.tasks (and the same names on the async client). They call the Tasks REST API endpoints for removing finished tasks, listing input/output storage files, and deleting files while keeping the task record.
1# Remove the task record (files stay in storage)2client.tasks.delete(task_id)34# Delete the task and its input/output files (task must be terminal)5client.tasks.delete(task_id, files=True)67# List platform files the task used or produced8files = client.tasks.files(task_id).data9outputs = client.tasks.files(task_id, role="output").data1011# Delete files but keep the task (terminal state required)12result = client.tasks.delete_files(task_id).data13print(result["deleted"], result["skipped"])14client.tasks.delete_files(task_id, role="input")role must be "input" or "output" when set. delete_files() returns a Response whose data matches DeleteTaskFilesResponse (deleted file IDs and skipped entries with reason). See List task files and Delete task files for resolution rules and error codes.
Inspecting tasks (logs, timings, telemetry)
The Python SDK does not yet expose getLogs, getTimings, or getTelemetry helpers. Call the REST endpoints directly:
1import httpx23API = "https://api.inference.sh"4headers = {5 "Authorization": "Bearer inf_your_key",6}78logs = httpx.get(f"{API}/tasks/{task_id}/logs", headers=headers).json()9timings = httpx.get(f"{API}/tasks/{task_id}/timings", headers=headers).json()10telemetry = httpx.get(f"{API}/tasks/{task_id}/telemetry", headers=headers).json()From the terminal: belt task logs <id>, belt task timings <id>. For telemetry, use the REST endpoint above or client.tasks.getTelemetry(id) in the JavaScript SDK.
→ Tasks REST API · Observability · SDK overview — inspect and observability
Agent SDK
Chat with AI agents programmatically.
Using an Existing Agent
Reference agents by namespace/name and version:
1from inferencesh import inference23client = inference(api_key="inf_your_key")45# Use an agent you created in the web app6agent = client.agent("my-org/my-agent@abc123") # Format: namespace/name@version78response = agent.send_message("Hello!")9print(response)Template agents that declare context fields take values when you create the agent (the agent's tools use them as {{context.NAME}}). Pass the context keyword argument (not available on client.agents.create()):
1agent = client.agent(2 "my-team/pricing-agent@latest",3 context={"version_id": "v1", "app_id": "app_123"},4)→ Template agents — per-chat context
Messaging channel routing
The API ignores channel_context on agent runs. In inferencesh ≥ v0.24.0, send_message() still accepts a channel_context argument for compatibility, but it is deprecated: passing it emits a DeprecationWarning and the value is not sent. Older versions sent it, and the run went ahead as if it were absent. Don't pass it. A chat's reply routing (Slack thread, Telegram chat, …) comes only from messages that arrive through the platform's channel integrations. Read it from agent.get_chat() on a channel-originated chat; see Get chat: channel_context.
Creating Ad-Hoc Agents
Create agents on-the-fly without saving to workspace:
1from inferencesh import inference, AdHocAgentOptions, tool, string23client = inference(api_key="inf_your_key")45# Define tools6calculator = (7 tool("calculator")8 .describe("Performs math operations")9 .param("expression", string("Math expression"))10 .build()11)1213# Create ad-hoc agent14agent = client.agent(AdHocAgentOptions(15 core_app={"ref": "infsh/claude-sonnet-4@latest"},16 name="Math Assistant",17 system_prompt="You help with math. Use the calculator tool.",18 tools=[calculator],19))2021response = agent.send_message("What is 42 * 17?")Streaming Responses
1def handle_message(msg):2 if msg.get("content"):3 print(msg["content"], end="", flush=True)45def handle_tool(call):6 print(f"\n[Tool: {call.name}]")7 # Execute tool and return result string8 result = execute_my_tool(call.name, call.args)9 agent.submit_tool_result(call.id, result)1011 # Return files with the result (inferencesh >= v0.27.3):12 # agent.submit_tool_result(call.id, result, attachments=[ref])13 14 # For widget interactions, you can also pass action + form_data:15 # agent.submit_tool_result(call.id, {16 # "action": {"type": "confirm"},17 # "form_data": {"name": "John", "email": "[email protected]"}18 # })1920response = agent.send_message(21 "Explain quantum computing",22 on_message=handle_message,23 on_tool_call=handle_tool,24)File Attachments
1# From file path2response = agent.send_message(3 "What's in this image?",4 files=[open("image.png", "rb").read()]5)67# From base648response = agent.send_message(9 "Analyze this",10 files=["data:image/png;base64,iVBORw0KGgo..."]11)send_message() uploads each files entry automatically. Use agent.upload_file() when you need the FileRef to reuse the same URI across messages:
1ref = agent.upload_file(open("image.png", "rb").read(), filename="image.png")2# ref["uri"] — use client.files or REST APIs if you need the URI outside send_message3response = agent.send_message("Summarize this file", files=[open("image.png", "rb").read()])Tool result attachments
inferencesh ≥ v0.27.3: submit_tool_result(tool_invocation_id, result, attachments=None) takes a list of FileRefs (uri, filename, content_type, optional id and size) and sends them on POST /tools/{id}. Upload the bytes first with agent.upload_file(). Each ref needs a uri; the API takes at most 20. AsyncAgent.submit_tool_result has the same signature.
1shot = agent.upload_file(open("screenshot.png", "rb").read(), filename="screenshot.png")2agent.submit_tool_result(call.id, "Captured the visible tab", attachments=[shot])Structured output (agent.run)
When the agent uses the finish tool, call agent.run() to send a message, wait for completion, and return the parsed finish-tool payload:
1result = agent.run("Summarize this document in JSON")2print(result)Equivalent to send_message() followed by reading chat["output"]. For streaming or tool callbacks, use send_message() instead.
Chat Management
1# Get current chat (metadata — status, output, active_run)2chat = agent.get_chat()3print(f"Chat ID: {chat['id']}")4print(f"Status: {chat['status']}")56# Message history: GET /chats/{id}/messages — see Agents REST API78# Stop generation9agent.stop_chat()1011# Start fresh12agent.reset()Tool Builder
Define tools with the fluent API:
1from inferencesh import (2 tool, app_tool, agent_tool, webhook_tool, internal_tools,3 string, number, integer, boolean, enum_of, array, obj, optional4)56# Client tool (runs in your code)7search = (8 tool("search_files")9 .describe("Search for files")10 .param("pattern", string("Glob pattern"))11 .param("recursive", optional(boolean("Search subdirs")))12 .build()13)1415# App tool (calls another inference app)16generate = (17 app_tool("generate", "infsh/flux-schnell@latest")18 .describe("Generate an image")19 .param("prompt", string("Image description"))20 .build()21)2223# Agent tool (delegates to sub-agent)24researcher = (25 agent_tool("research", "my-org/researcher@v1")26 .describe("Research a topic")27 .param("topic", string("Topic"))28 .build()29)3031# Webhook tool (calls external API)32notify = (33 webhook_tool("slack", "https://hooks.slack.com/...")34 .describe("Send Slack message")35 .secret("SLACK_SECRET")36 .param("message", string("Message"))37 .build()38)3940# Call tool — authenticated HTTP (runtime injects vault secrets or OAuth tokens)41from inferencesh import call_tool4243calendar = (44 call_tool("list_events", "https://www.googleapis.com/calendar/v3/calendars/primary/events")45 .method("GET")46 .auth(credential="google")47 .describe("List calendar events")48 .build()49)5051# MCP connector tool — the server by slug; it runs as the caller's own login52from inferencesh import mcp_tool5354list_issues = (55 mcp_tool("list_issues", server="linear", tool_name="list_issues")56 .describe("List Linear issues")57 .build()58)5960# Internal tools config (default-on categories: plan, memory, widget, finish, skills)61internals = internal_tools().memory().plan().build()6263# Opt-in categories: chain builders (inferencesh >= v0.14.1)64internals = (65 internal_tools()66 .meta()67 .artifact()68 .agent()69 .remote()70 .knowledge()71 .build()72)7374# Disable skill_get while enabling knowledge tools75internals = internal_tools().skills(False).knowledge().build()internal_tools().all() enables plan, memory, widget, and finish only; it does not turn on opt-in categories (meta, artifact, agent, remote, knowledge). The serialized field is agent, not spawn; older agent versions may still store "spawn": true, which the API treats as agent.
→ Internal tools: defaults, tool names, and category reference
call_tool().auth(credential=...) writes type: "credential" with provider in the serialized tool; each call signs in with the caller's own login to that provider. inferencesh v0.32.0 removed credential_id= from .auth() and mcp_tool() (see Call tools).
→ Call tools · Connector tools · Full Tool Builder Reference
App authoring (LLM inputs)
When you build Python model apps, import typed chat inputs from inferencesh.models. As of v0.7.9, use ChatInput with a nested ModelSettings object for sampling parameters instead of flat top-level fields.
1from inferencesh.models import ChatInput, ModelSettings, LLMOutput, BaseApp23class MyInput(ChatInput):4 pass56class App(BaseApp):7 async def run(self, input_data: MyInput, metadata) -> LLMOutput:8 # input_data.text, input_data.context, input_data.system_prompt9 # input_data.model_settings.temperature, .max_tokens, .stop, etc.10 ...ModelSettings groups provider-agnostic generation knobs. Every field is optional; omitted fields are not sent to the provider and the app’s defaults apply.
| Field | Description |
|---|---|
temperature | Sampling temperature (0–2) |
top_p | Nucleus sampling probability mass |
top_k | Top-k sampling (-1 disables) |
min_p | Min-p sampling cutoff |
frequency_penalty | Penalize tokens by how often they appear |
presence_penalty | Penalize tokens that already appeared |
repetition_penalty | Multiplicative repetition penalty (vLLM/HuggingFace style) |
seed | Random seed for reproducible output |
stop | Stop sequences |
max_tokens | Maximum tokens to generate |
Example task input:
1{2 "text": "Write a haiku about clouds",3 "model_settings": {4 "temperature": 0.8,5 "max_tokens": 256,6 "stop": ["\n\n"]7 }8}Backwards compatibility: LLMInput (subclass of BaseLLMInput) keeps flat temperature, top_p, and max_tokens for existing apps. New apps should prefer ChatInput.
Add only the nested settings field to a custom input with ModelSettingsCapabilityMixin. Capability mixins (ImageCapabilityMixin, ReasoningCapabilityMixin, ToolsCapabilityMixin, and others) live in inferencesh.models.llm.
Helpers for OpenAI-style message lists and streaming: build_messages, stream_generate, timing_context (import from inferencesh or inferencesh.models).
build_messages is an alias of build_openai_messages(input_data, ..., image_limits=None, tool_media_as_user_message=False) (in inferencesh.models.llm):
image_limits: anImageLimitsfor the provider. Base64 images are downsized to fit.tool_media_as_user_message: by default a tool's images and files go in its role"tool"message. SetTruefor a backend that accepts only text there; they then go in a user message after the tool messages.reasoning_state_key(≥ v0.27.1): the key your app writesreasoning_detailsunder inreasoning_state. An assistant message with an entry under that key sends it back asreasoning_details; entries under other keys are left out.reasoning_field(≥ v0.27.4): the field an assistant message sends its ownreasoningtext back on, for a provider that takes it as text (DeepSeek and the backends serving it usereasoning_content). Applies when the message has no entry underreasoning_state_key. Without it,reasoningfromcontextis not sent.include_reasoning=Trueis deprecated; it is the same asreasoning_field="reasoning".
Reasoning state (inferencesh ≥ v0.27.0). LLMOutput and ContextMessage have an optional reasoning_state: a map of opaque JSON a provider needs on the next request (Anthropic thinking signatures, Gemini thought_signature, OpenAI encrypted reasoning). Set it on the final LLMOutput under a key your app owns; LLMDelta has no such field. The agent stores it on the assistant message and sends it back on that message's context entry on later turns, including in forked chats. It is not on chat messages from the REST API or SSE. Place only your own entry in a provider request: another provider's signature is refused.
1from inferencesh.models import LLMOutput, build_messages2from inferencesh.models.llm import merge_reasoning_details34# End of a turn: fold streamed reasoning_details fragments into whole blocks5output = LLMOutput(6 response=text,7 reasoning_state={"openrouter": merge_reasoning_details(streamed_details)},8)910# Next turn: each prior assistant message sends its "openrouter" entry back11messages = build_messages(input_data, reasoning_state_key="openrouter")1213# DeepSeek: each assistant message carries its own reasoning text14messages = build_messages(input_data, reasoning_field="reasoning_content")merge_reasoning_details (≥ v0.27.1) joins consecutive fragments with the same index and type, concatenating text, summary and data and keeping the other fields.
inferencesh ≥ v0.26.0 adds inferencesh.images for fitting an image to a provider's limits:
1from inferencesh.images import ImageLimits, fit_image23data, mime = fit_image(4 "/tmp/screenshot.png",5 ImageLimits(max_long_edge=2048, max_pixels=1_150_000, max_bytes=5_000_000, mime_types=("image/png", "image/jpeg")),6)fit_image returns (bytes, mime), downsizing only; it never upscales. Each limit is optional (None means no limit of that kind). It needs Pillow: pip install "inferencesh[images]". Without Pillow it logs a warning and returns the original bytes.
inferencesh ≥ v0.13.0 merges streaming deltas with the same rules as the JavaScript SDK and belt: fields tagged in _field_tags on LLMDelta (and related delta types) use concat, replace, indexed, or nested; untagged strings concatenate and other untagged values replace. Custom pydantic delta types should set _field_tags explicitly when you stream partial objects outside stream_generate().
App authoring (decision models)
Decision apps (AppCategory.DECISION) use a fixed request/response shape so callers can swap models without changing integration code. Pydantic helpers ship in inferencesh ≥ v0.20.0 (inferencesh.models, backed by the generated API contract in llm_types_gen.py):
1from inferencesh import BaseApp2from inferencesh.models import DecisionInput, DecisionOutput, DecisionVisionInput, DecisionVideoInput3from inferencesh.models.decision import (4 ChoiceQuestion,5 ScoreQuestion,6 NoulQuestion,7 ChoiceAnswer,8 ScoreAnswer,9 NoulAnswer,10)1112class App(BaseApp):13 async def run(self, input_data: DecisionInput, metadata) -> DecisionOutput:14 ...| Type | Purpose |
|---|---|
DecisionInput | state plus optional choices, scores, nouls question lists |
DecisionVisionInput | DecisionInput plus images (file URIs shared by every question) |
DecisionVideoInput | DecisionVisionInput plus videos (file URIs every question sees after the images; inferencesh ≥ v0.23.0) |
DecisionOutput | Maps of answers under choices, scores, nouls keyed by question id; plus model, input_tokens |
ChoiceQuestion / ScoreQuestion / NoulQuestion | Question builders on DecisionInput (import from inferencesh.models.decision) |
ChoiceAnswer | choice, confidence, probabilities |
ScoreAnswer | score, normalized, confidence, probabilities, legend |
NoulAnswer | noul (probability yes) |
App authors that wrap a shared decision model can build outputs with DecisionOutput.from_answers(answers, input_data, model=..., input_tokens=...) where answers matches the wire shape keyed by question id.
Field semantics, JSON examples, and authoring notes: App code — Decision models.
App authoring (function metadata)
function declares a function's credentials and tool annotations. It stores them on the method and returns the method unchanged.
1from inferencesh import BaseApp, function23class App(BaseApp):4 @function(5 credentials=[{"key": "google.calendar.events"}],6 annotations={"readOnlyHint": False, "destructiveHint": False},7 )8 async def create_event(self, input_data: CreateEventInput) -> CreateEventOutput:9 ...Both arguments are keyword-only and optional. credentials takes entries shaped like inf.yml credentials: entries (CredentialRequirement); annotations takes MCP tool annotations (ToolAnnotations). Both types are in inferencesh.types.
→ Function metadata · Per-function credentials
Async Support
The async client mirrors the sync namespaced API (tasks, files, agents, sessions).
1from inferencesh import async_inference, TaskStatus2import asyncio34async def main():5 client = async_inference(api_key="inf_...")67 # Run and wait (default)8 result = await client.tasks.run({"app": "infsh/echo", "input": {...}})910 # Fire and forget, then wait or stream11 task = await client.tasks.run({"app": "infsh/slow-task", "input": {...}}, wait=False)12 result = await client.tasks.wait_for_completion(task["id"], timeout=300)1314 async with client.tasks.stream(task["id"]) as stream:15 async for update in stream:16 if update["status"] == TaskStatus.COMPLETED:17 break1819 await client.tasks.delete(task["id"], files=True)20 await client.tasks.files(task["id"])21 await client.tasks.delete_files(task["id"])2223 # Upload via files namespace24 file_obj = await client.files.upload(b"PNGDATA")2526 # Async agent27 agent = client.agent("my-org/assistant@abc123")28 response = await agent.send_message("Hello!")2930 async for msg in agent.stream_messages():31 print(msg)3233asyncio.run(main())client.run() and client.upload_file() work as legacy aliases on the async client as well.
Error Handling
With API v2, HTTP failures return RFC 9457 application/problem+json. The sync and async clients parse the same fields: APIError.message prefers detail, then title, then message (for non-standard problem payloads), and falls back to the raw body for non-JSON responses.
1from inferencesh import APIError, RequirementsNotMetError23try:4 result = client.tasks.run({"app": "my-app", "input": {...}})5except RequirementsNotMetError as e:6 # HTTP 412 — requirements payload is not wrapped in v27 print(f"Missing requirements: {e.errors}")8 for err in e.errors:9 # err["type"] is "credential" (RequirementType.CREDENTIAL); "integration" still appears on older payloads10 print(f" - {err['type']}: {err['key']}")11except APIError as e:12 print(f"HTTP {e.status_code}: {e.message}") # e.response_body has full problem+json13except RuntimeError as e:14 print(f"Error: {e}")async_inference() raises the same exception types with the same message extraction. Session helpers may raise SessionNotFoundError, SessionExpiredError, or SessionEndedError (subclasses of APIError).
inferencesh ≥ v0.13.0 exports ErrorCode and typed meta objects for common problem responses. The problem body has no code field: the code is the last segment of type (https://api.inference.sh/errors/payment_method_required). inferencesh ≥ v0.21.0 reads it for you as APIError.code:
1import json2from inferencesh import APIError3from inferencesh.types import (4 ErrorCode,5 EntitlementErrorMeta,6 PaymentMethodRequiredMeta,7 TeamRoleRequiredMeta,8)910try:11 client.tasks.run({"app": "my-app", "input": {...}})12except APIError as e:13 problem = json.loads(e.response_body)14 code = e.code # before v0.21.0: problem["type"].rsplit("/", 1)[-1]15 meta = problem.get("meta") or {}1617 if code == ErrorCode.TEAM_ROLE_REQUIRED.value:18 role_meta: TeamRoleRequiredMeta = meta19 print(role_meta.get("capability"), role_meta.get("required_role"))20 elif code == ErrorCode.PAYMENT_METHOD_REQUIRED.value:21 pay_meta: PaymentMethodRequiredMeta = meta22 print(pay_meta.get("billing_page"))23 elif code in (ErrorCode.LIMIT_EXCEEDED.value, ErrorCode.FEATURE_NOT_AVAILABLE.value):24 ent_meta: EntitlementErrorMeta = meta25 if ent_meta.get("upgrade_available"):26 print(ent_meta.get("addon_plan_name"))A call that needs the account holder's own sign-in (tool approvals, for example) refuses an API key with 403 requires_sign_in (ErrorCode.REQUIRES_SIGN_IN, inferencesh ≥ v0.21.0).
Remote harness refusals (remote_offline, harness_not_drivable, harness_too_old, …) use the same ErrorCode values — see Agents REST — Remote harness runs.
→ REST overview — response format — problem+json shape
Generated types (inferencesh.types)
The package ships OpenAPI-aligned types for REST integrations. Import from inferencesh.types (not the top-level inferencesh package) when you call endpoints that do not yet have a Python client namespace.
MCP tasks extension — Custom MCP clients that call inference.sh over Streamable HTTP should declare the tasks extension and interpret resultType on tools/call responses. inferencesh ≥ v0.13.0 adds ClientCapabilities.extensions and ResultType.TASK for async task handles (for example long-running app_run):
1from inferencesh.types import ClientCapabilities, ResultType, ToolCallResponse23client_caps: ClientCapabilities = {4 "extensions": {"io.modelcontextprotocol/tasks": {}},5}67response: ToolCallResponse = {...} # from tools/call8if response.get("resultType") == ResultType.TASK:9 ... # CreateTaskResult includes taskId — poll tasks/get until completed10elif response.get("resultType") == ResultType.INPUT_REQUIRED:11 ... # MRTR — retry with inputResponses (see using-tools MRTR section)Wire protocol steps (capabilities JSON, tasks/get, tasks/cancel) are in MCP Server — Tasks extension and app_run. MRTR elicitation uses ResultType.INPUT_REQUIRED on the same ToolCallResponse shape.
Tool parameter schema (ToolParameterProperty) — Agent and app tools serialize parameters with the same shape the platform sends to model providers. inferencesh ≥ v0.14.0 adds optional anyOf and enum on ToolParameterProperty in inferencesh.types and on the pydantic mirror in inferencesh.llm_types_gen (aligned with agentprotocol v0.16.0):
1from inferencesh.types import ToolParameterProperty, ToolParamType23# Union parameter: value may be a string or an integer4status_code: ToolParameterProperty = {5 "title": "status_code",6 "description": "HTTP status as text or number",7 "anyOf": [8 {"type": ToolParamType.STRING, "title": "status_code"},9 {"type": ToolParamType.INTEGER, "title": "status_code"},10 ],11}1213# Closed set on a scalar (also duplicated in description text for legacy readers)14sort: ToolParameterProperty = {15 "type": ToolParamType.STRING,16 "title": "sort",17 "enum": ["count", "first_seen"],18}When anyOf is set, omit type on the parent property — a single type would misrepresent a multi-shape field. MCP connector tools populate these fields when converting cached MCP inputSchema JSON; see Connector tools — unions and enums.
Agent run wire types: inferencesh ≥ v0.22.0 types POST /agents/run as ApiAgentRunRequest, with the user turn in input: MessageInput (text, attachments; the deprecated images / files are read only when attachments is empty). CreateAgentMessageRequest is no longer generated. ContextField.in_prompt: true shows a context value to the model, quoted in the system prompt; otherwise the model only learns the field's name and writes {{context.NAME}} in tool arguments.
1from inferencesh.types import ApiAgentRunRequest, MessageInput, ContextField23turn: MessageInput = {"text": "Summarize this", "attachments": []}4run: ApiAgentRunRequest = {5 "agent": "my-org/support-agent",6 "input": turn,7 "context": {"project_id": "proj_456"},8}910plan_field: ContextField = {11 "name": "plan",12 "description": "Billing plan label for the caller",13 "in_prompt": True,14}agent.send_message() builds the same body. See Adding tools: Declaring context fields.
Messaging channels: ChannelContext on ChatDTO when a chat originated from Slack, Telegram, or another platform:
1from inferencesh.types import ChannelContext, ChannelType23ctx: ChannelContext = {4 "channel_type": ChannelType.SLACK,5 "channel_metadata": {"channel": "C01234567", "thread_ts": "1234567890.123456"},6}channel_metadata is provider-specific opaque JSON for reply routing. See Messaging channel routing and Get chat — channel_context.
Chat settings and agent switch — POST /chats/{chatId}/settings and POST /chats/{chatId}/agent return narrow DTOs (ChatSettingsDTO, ChatAgentDTO) that describe what changed — not a full ChatDTO. Merge them into the chat you hold. There is no Python client helper yet; call the REST endpoints and type requests and responses with inferencesh.types (inferencesh ≥ v0.16.0 for the response DTOs; ChatSettingsRequest ships from v0.14.3):
1import httpx2from inferencesh.types import (3 ChatSettingsRequest,4 ChatSettingsDTO,5 ChatAgentDTO,6)78settings_body: ChatSettingsRequest = {9 "name": "Incident triage",10 "visibility": "team",11 "allow_all_tools": False,12 "disable_hooks": True,13 "forget_memory": ["stale_goal"],14}15settings_resp = httpx.post(16 f"{API}/chats/{chat_id}/settings",17 headers={"Authorization": f"Bearer {api_key}"},18 json=settings_body,19)20updated_settings: ChatSettingsDTO = settings_resp.json()["data"]2122switch_resp = httpx.post(23 f"{API}/chats/{chat_id}/agent",24 headers={"Authorization": f"Bearer {api_key}"},25 json={"agent": "myteam/escalation-agent@latest"},26)27agent_change: ChatAgentDTO = switch_resp.json()["data"]Omit keys you do not want to change. disable_hooks stops lifecycle hooks in that chat (review branches are opened with it set). Responses use the standard envelope, so read data. The settings response is a ChatSettingsDTO (chat_id, name, visibility, allow_all_tools, disable_hooks, memory) — merge those fields onto your local chat (map the three agent_data fields) instead of replacing stored messages. POST /chats/{id}/agent returns ChatAgentDTO with the resolved agent refs. See Update chat settings and Set agent.
Scoped always allow and explain — when a platform-loop tool pauses with awaiting_approval, list scoped always-allow choices, save one (which also approves the current call), or fetch a one-time plain-language summary. There is no Python client helper yet (the JavaScript Agent SDK wraps these). Call the REST endpoints and type payloads with inferencesh.types (inferencesh ≥ v0.16.0):
1import httpx2from inferencesh.types import (3 AlwaysAllowOptionsDTO,4 AlwaysAllowRequest,5 AlwaysAllowResultDTO,6 ToolExplanationDTO,7)89options: AlwaysAllowOptionsDTO = httpx.get(10 f"{API}/chats/{chat_id}/tools/{tool_invocation_id}/always-allow/options",11 headers={"Authorization": f"Bearer {api_key}"},12).json()["data"]1314key = options.get("default") or options["options"][0]["key"]15body: AlwaysAllowRequest = {"option": key}16saved: AlwaysAllowResultDTO = httpx.post(17 f"{API}/chats/{chat_id}/tools/{tool_invocation_id}/always-allow",18 headers={"Authorization": f"Bearer {api_key}"},19 json=body,20).json()["data"]2122explanation: ToolExplanationDTO = httpx.post(23 f"{API}/chats/{chat_id}/tools/{tool_invocation_id}/explain",24 headers={"Authorization": f"Bearer {api_key}"},25).json()["data"]| REST endpoint | Types |
|---|---|
GET .../always-allow/options | AlwaysAllowOptionsDTO, AlwaysAllowOptionDTO, PolicyRuleDTO |
POST .../always-allow | AlwaysAllowRequest → AlwaysAllowResultDTO |
POST .../explain | ToolExplanationDTO |
Enums: AlwaysAllowScope (exact, prefix, folder, remote, tool), ToolRiskLevel (low, medium, high), PolicyEffect (allow, ask, deny). Re-fetch options after HTTP 409 (stale option key). explain is not available on harness-backed chats.
Policy kinds (inferencesh ≥ v0.17.0) — saved always-allow choices and workspace policy rows share PolicyRuleDTO. kind is typed as PolicyKind (use the enum instead of raw strings):
PolicyKind member | API value | Governs |
|---|---|---|
REMOTE_EXEC | RemoteExec | Shell commands on a remote (remote_exec) |
WORKSPACE | Workspace | Folders on a remote |
HARNESS | Harness | Harness tool approvals (for example remote_session_resolve) |
TOOL | Tool | Platform-loop tool calls by tool name |
WEB_FETCH | WebFetch | Outbound HTTP fetch domains |
APP, AGENT, KNOWLEDGE, MCP, FLOW | App, Agent, … | Usage policy (resolved resource ids) |
1from inferencesh.types import AgentPermissions, PolicyKind, PolicyRuleDTO23def describe_rule(rule: PolicyRuleDTO) -> str:4 if rule.get("kind") == PolicyKind.REMOTE_EXEC:5 return f"remote command: {rule.get('specifier')}"6 if rule.get("kind") == PolicyKind.TOOL:7 return f"tool: {rule.get('specifier')}"8 return f"{rule.get('kind')} {rule.get('specifier')}"910defaults: AgentPermissions = {"allow_all_tools": True}Agent permissions — AgentPermissions (allow_all_tools optional) types version.permissions on agent templates. It seeds agent_data.allow_all_tools when a new chat is created in the agent's own workspace; chats other workspaces start with the agent get none of these defaults (they run on their own remotes and tools). Team and org policy can still ask or deny over it. See Allow all tools on new chats (agent template).
inferencesh ≥ v0.17.0 removed ChatData.always_allowed_tools from generated types — always-allow is stored as chat permission rules. Read rules from AlwaysAllowResultDTO, call GET /chats/{id}/rules, or use chat settings. ChatData still includes allow_all_tools and disable_hooks. Legacy chats may return always_allowed_tools on the wire until migrated; the REST docs describe synthesized rule rows.
→ List always-allow options · Explain tool call · Chat permissions rules · Approval settings
Knowledge — use with the Knowledge REST API until client.knowledge ships in Python (available today in the JavaScript SDK):
1from inferencesh.types import KnowledgeType, KnowledgeLifecycle, KnowledgeDTO23# Enum values match POST /knowledge `type` and `lifecycle` fields4KnowledgeType.OBSERVATION # "observation"5KnowledgeType.SKILL # "skill" — prefer /skills endpoints when applicable6KnowledgeLifecycle.DECAY # "decay"7KnowledgeLifecycle.PERMANENT # "permanent"| Enum | Members (API string values) |
|---|---|
KnowledgeType | concept, skill, observation, preference, reference, person, project, agent-config |
KnowledgeLifecycle | permanent, decay |
KnowledgeDTO, KnowledgeCreateRequest, and related request types mirror the REST schemas for typed httpx or requests calls. Set scope on KnowledgeVersionInput (or version.scope on create) to tag environment signals — see Knowledge API — Scoping.
Graph edges — edge type values on knowledge references, skill lineage, and Supersede Skill:
1from inferencesh.types import GraphEdgeType23GraphEdgeType.REFERENCES # "references" — content mentions in markdown4GraphEdgeType.SUPERSEDES # "supersedes" — replacement skill hides the old one from suggest| Enum | Members (API string values) |
|---|---|
GraphEdgeType | dependency, flow, conditional, execution, parent, ancestor, duplicate, references, supersedes |
Search / suggest — use with the Search REST API until client.search ships in Python (available today in the JavaScript SDK):
1from inferencesh.types import SuggestRequest, SuggestResponse, SuggestResult23request: SuggestRequest = {4 "query": "image generation",5 "context": "User asked for a fast model for social thumbnails",6 "limit": 5,7 "category": "apps",8 "scope": ["git:github.com/acme/design", "lang:typescript"],9}10result: SuggestResult = {11 "type": "app",12 "name": "okaris/flux",13 "description": "Image generation",14 "command": "inference run okaris/flux@abc1",15 "score": 0.92,16}17knowledge_hit: SuggestResult = {18 "type": "knowledge",19 "tag": "reference",20 "name": "myteam/stripe-webhooks",21 "description": "Stripe webhook handling patterns",22 "command": "belt knowledge get myteam/stripe-webhooks",23 "score": 0.81,24}25response: SuggestResponse = {26 "query": "image generation",27 "results": [result, knowledge_hit],28}For hybrid ranking, the API embeds query, then optional context, then scope tags. Keyword matching uses query only — see Search API — Suggest.
| TypedDict | Fields |
|---|---|
SuggestRequest | query, context, limit, category, scope (agent is deprecated — ignored) |
SuggestResponse | query, results (List[SuggestResult]) |
SuggestResult | type, tag, name, description, command, score (descriptions clipped to 200 characters) |
inferencesh ≥ v0.19.0 exports DescriptionLimit (IntEnum) with LISTING (200) and SKILL (1024) for client-side validation before writes — see description length limits:
1from inferencesh.types import DescriptionLimit23assert DescriptionLimit.LISTING == 2004assert DescriptionLimit.SKILL == 1024Suggest result description fields are clipped to DescriptionLimit.LISTING characters on read.
Instance catalog — use with Instance types REST API for GPU pricing and availability:
1from inferencesh.types import (2 InstanceRentalType,3 InstanceTypeAvailability,4 InstanceTypeDTO,5 InstanceTypeConfiguration,6)78# GET /instances/types returns a list of InstanceTypeDTO9row: InstanceTypeDTO = {10 "cloud": "runpod",11 "cloud_logo_url": "https://cloud.inference.sh/logos/runpod.com.png",12 "configuration": {13 "gpu_type": "H100",14 "gpu_manufacturer": "nvidia",15 "num_gpus": 1,16 "nvlink": False,17 },18 "availability": [19 {"available": True, "region": "us-east-1", "rental_type": InstanceRentalType.ON_DEMAND},20 {21 "available": True,22 "region": "us-east-1",23 "rental_type": InstanceRentalType.SPOT,24 "hourly_price": 199, # cents per hour — spot rows only25 },26 ],27}28spot_row: InstanceTypeAvailability = row["availability"][1]InstanceTypeConfiguration includes gpu_manufacturer, nvlink, gpu_type, num_gpus, and the other fields documented on the REST page. Each availability entry includes rental_type (on_demand or spot); spot rows also set hourly_price (cents). Engine-picker offers from GET /engines/types set top-level rental_type, region, and hourly_price on InstanceTypeDTO, and include an options array (every in-stock provider and per-region prices for that offer) — see Instance types and Engines API — List engine instance types.
Store browse metadata — category and tag catalogs from the public Store API (StoreCategoryDTO / StoreTagDTO ship in the next inferencesh release after v0.20.0; until then, read the JSON as plain dicts):
1import httpx2from inferencesh.types import StoreCategoryDTO, StoreTagDTO34categories: list[StoreCategoryDTO] = httpx.get(5 "https://api.inference.sh/store/categories",6 params={"type": "app"},7).json()["data"]89tags: list[StoreTagDTO] = httpx.get("https://api.inference.sh/store/tags").json()["data"]| TypedDict | REST endpoint | Notes |
|---|---|---|
StoreCategoryDTO | GET /store/categories | slug, name, description, icon, rank, count — pass type to scope count per resource kind |
StoreTagDTO | GET /store/tags | slug, title, count — tags on ≥ 3 public apps |
There is no client.store namespace yet; call these endpoints with httpx or requests. See List store categories and List browseable tags.
Apps — type GET /apps/{namespace}/{name} responses with AppDTO when you call the Apps REST API directly. Ref-route aliases return the target app's namespace, name, and id; when the requested name pins a multi-function entry point, resolved_function names that function (omitted when the caller should use the version default). resolved_function is on AppDTO from inferencesh ≥ v0.19.0 (JavaScript: @inferencesh/sdk ≥ v0.20.0):
1import httpx2from inferencesh.types import AppDTO34resp = httpx.get(5 "https://api.inference.sh/apps/elevenlabs/text-to-dialogue",6 headers={"Authorization": "Bearer inf_..."},7)8app: AppDTO = resp.json()["data"]9entry = app.get("resolved_function") or "default"Per-app store listings — use with the Store REST API:
1from inferencesh.types import AppStoreListingDTO23listing: AppStoreListingDTO = {4 "id": "app_abc123",5 "allows_cloud_workers": True,6 "allows_private_workers": True,7 "max_concurrency": 0,8 "max_concurrency_per_team": 5,9 "min_concurrency": 2,10 "required_feature": "feature:seedance", # optional — explicit entitlement grant required11}min_concurrency protects cloud workers holding the app from displacement by other apps; it does not start workers. See Reserved cloud capacity. When required_feature is set, dispatch requires a boolean entitlement with enabled: true — see Feature-gated store apps.
App metadata
App listing fields on GET /apps/{id} and create/deploy payloads (inferencesh ≥ v0.19.1 for AppCategory.DECISION and tags on AppDTO / CreateAppRequest):
1from inferencesh.types import AppCategory, AppDTO, CreateAppRequest23app: AppDTO = {4 "namespace": "myteam",5 "name": "router",6 "category": AppCategory.DECISION,7 "tags": ["routing", "reasoning"],8}910create: CreateAppRequest = {11 "namespace": "myteam",12 "name": "flux-schnell",13 "category": AppCategory.IMAGE,14 "tags": ["text-to-image", "open-weights"],15}1617# Browseable slugs and display titles (public, no API key)18import httpx1920tags = httpx.get("https://api.inference.sh/store/tags").json()["data"]21titles = {row["slug"]: row["title"] for row in tags}22label = titles.get("text-to-image", "text-to-image")| Enum / TypedDict | Notes |
|---|---|
AppCategory | image, video, audio, text, chat, 3d, flow, decision, other — decision is for probability-over-answers models, not generative output |
AppDTO | Response from GET /apps/...; includes tags when set and optional resolved_function on ref lookups (see Apps API — Get app) |
CreateAppRequest | POST /apps body; optional tags replaces the app's tags when provided |
Browseable tag slugs and titles come from GET /store/tags (Store API). v0.19.1 briefly exported AppTag / AppTagTitle enums; current types use string[] tags only so the tag set can change without a new SDK release.
Tag rules (slug format, max count, deploy vs metadata.tags) are documented on the Apps API.
Bounty types: use with Workspaces API: bounties (GET /bounties, POST /me/bounty). There is no client.bounties namespace yet; import the generated types when calling these endpoints with httpx or requests:
1import httpx2from inferencesh.types import (3 BountyProgramDTO,4 SubmitBountyRequest,5 SubmitBountyResponse,6)78programs: list[BountyProgramDTO] = httpx.get(9 f"{base_url}/bounties",10 headers={"Authorization": f"Bearer {api_key}"},11).json()1213claim: SubmitBountyRequest = {14 "bounty_id": "app-builder",15 "proof_id": "01abc123def456", # app id or namespace/name when proof_type is app16 "source": "cli",17}1819# Form programs: submit the form first, then claim with proof_id = submission.id20form_claim: SubmitBountyRequest = {21 "bounty_id": "agent-survey",22 "proof_id": "fsm_abc123",23 "source": "cli",24}2526result: SubmitBountyResponse = httpx.post(27 f"{base_url}/me/bounty",28 headers={29 "Authorization": f"Bearer {api_key}",30 "Content-Type": "application/json",31 },32 json=claim,33).json()| TypedDict | Use |
|---|---|
BountyProgramDTO | Items returned by GET /bounties — includes proof_type (app, form, text, none) and proof_form (namespace/name when proof_type is form) |
SubmitBountyRequest | Body for POST /me/bounty — proof_id is an app ref/id, a form submission id, or a text key depending on the program |
SubmitBountyResponse | Response from POST /me/bounty |
BountySubmissionDTO | Nested in SubmitBountyResponse.submission — resource_id is the app or form ID when applicable; proof_id is the submission id for form proofs |
Duplicate claims return HTTP 409 with code already_claimed (ErrorCode.ALREADY_CLAIMED in generated types). When requires_payment_method is set, a missing card returns 402 and no claim row is stored (safe to retry after billing is set up).
Forms — use with the Forms REST API (GET /forms, POST /forms/{id}/submissions). There is no client.forms namespace yet; the types ship in inferencesh ≥ v0.19.0:
1from inferencesh.types import (2 FormDTO,3 SubmitFormRequest,4 SubmitFormResponse,5 FormSubmissionDTO,6)78submit: SubmitFormRequest = {9 "data": {"discovery": "github"},10 "source": "script",11 "agent": "cursor",12}1314recorded: SubmitFormResponse = httpx.post(15 f"{base_url}/forms/{form_id}/submissions",16 headers={17 "Authorization": f"Bearer {api_key}",18 "Content-Type": "application/json",19 },20 json=submit,21).json()22# recorded["submission"] is a FormSubmissionDTO — credits are not granted hereFormDTO has no bounty_name field; link programs through BountyProgramDTO.proof_form instead. SubmitSurveyResponse still includes granted_amount and reward_blocked_reason for older clients, but servers return 0 and "" — claim survey bounties with POST /me/bounty after POST /me/survey.
Cost estimation — use with Estimate app cost (POST /store/apps/{appId}/estimate):
1from inferencesh.types import EstimateCostRequest, EstimateCostResponse23req: EstimateCostRequest = {4 "input": {"n": 4},5 "function": "generate", # optional for multi-function apps6}7resp: EstimateCostResponse = {8 "confidence": "range",9 "min": 150000,10 "max": 400000,11 "depends_on": ["output_tokens"],12 "pricing_description": "$0.0015–$0.004 per run",13}14failed: EstimateCostResponse = {15 "confidence": "unknown",16 "depends_on": ["output_tokens"],17 "estimate_error": "CEL evaluation failed: undefined variable 'output_tokens'",18}| TypedDict | Fields |
|---|---|
EstimateCostRequest | input, function |
EstimateCostResponse | confidence, microcents, min, max, depends_on, estimate_error, pricing_description |
Plans and subscriptions — use with the Subscription REST API:
1from inferencesh.types import PlanDTO, PlanType, EntitlementResource23base: PlanDTO = {4 "name": "pro",5 "plan_type": PlanType.BASE,6 "credits_monthly": 5_000_000_000,7 "stackable": False,8 "required_plan_ids": [],9 "limits": {},10}11addon: PlanDTO = {12 "name": "extra concurrency",13 "plan_type": PlanType.ADDON,14 "stackable": True,15 "required_plan_ids": ["plan_pro", "plan_team"],16 "required_plan_names": ["Pro", "Team"],17 "limits": {18 EntitlementResource.RESOURCE_CONCURRENCY: {"limit": 5},19 },20}stackable controls whether a workspace can purchase multiple instances of the same add-on. required_plan_ids gates add-on checkout to workspaces on a qualifying base plan (see Add-on subscriptions).
API key scopes — use with API keys — Scope catalog (GET /scopes):
1from inferencesh.types import ScopesResponse, ScopePreset, ScopeDefinition23catalog: ScopesResponse = {4 "scopes": [5 {"value": "agents:execute", "label": "execute", "group": "agents"},6 ],7 "groups": [{"id": "agents", "label": "agents"}],8 "presets": [9 {10 "id": "run",11 "label": "run",12 "description": "execute agents, apps, and flows",13 "scopes": ["agents:read", "apps:read", "agents:execute", "apps:execute"],14 "summary": ["run apps and agents on your behalf"],15 "hidden": False,16 }17 ],18}ScopePreset.summary is a human-readable permission list for consent UIs. hidden presets (for example engine) are omitted from non-admin GET /scopes responses.
Current user (GET /me) — inferencesh ≥ v0.13.0 exports MeResponse and the nested DTOs the REST page documents (TeamDTO, OrgDTO, TeamViewDTO, DiagnosticsConfig, TeamKind). Use them with client helpers or direct HTTP:
1import httpx2from inferencesh.types import MeResponse, TeamDTO, UserMetadataDTO34me: MeResponse = httpx.get(5 f"{base_url}/me",6 headers={"Authorization": f"Bearer {api_key}"},7).json()["data"]89team: TeamDTO | None = me.get("team")10metadata: UserMetadataDTO = (me.get("user") or {}).get("metadata") or {}team is a full workspace record (TeamDTO), not the list-item shape from GET /teams. Capability checks belong on me["team_view"]["can"] (and me["team_view"]["org"]["can"] for org-workspace actions) — see Get current user.
User metadata — nested on UserDTO inside MeResponse:
1from inferencesh.types import UserMetadataDTO23metadata: UserMetadataDTO = {4 "completed_onboarding": True,5 "terms_accepted_at": "2026-01-15T10:30:00Z",6 "terms_version": "2026-01",7}Entitlements — use with the Entitlements REST API:
1from inferencesh.types import EntitlementDTO, EntitlementSource, EntitlementType, EntitlementResource23ent: EntitlementDTO = {4 "resource": EntitlementResource.RESOURCE_CONCURRENCY,5 "type": EntitlementType.LIMIT,6 "limit": 3,7 "source": EntitlementSource.TIER,8}Entitlement error metadata — EntitlementErrorMeta mirrors the meta object on limit_exceeded and feature_not_available responses. Parse it from APIError.response_body:
1import json2from inferencesh import APIError3from inferencesh.types import EntitlementErrorMeta45try:6 client.tasks.run({"app": "my-app", "input": {...}})7except APIError as e:8 body = json.loads(e.response_body)9 meta: EntitlementErrorMeta = body.get("meta") or body.get("error", {}).get("meta", {})10 if meta.get("upgrade_available"):11 print(meta.get("resource_label"), meta.get("addon_plan_name"))See Entitlements API — Error metadata for field definitions (addon_plan_id, addon_plan_price, upgrade_available, etc.).
Team settings capabilities — inferencesh ≥ v0.13.0 adds TeamCapability for the can arrays on GET /teams/{id}/view and team_view on GET /me:
1import httpx2from inferencesh.types import TeamCapability34view = httpx.get(5 f"{base_url}/teams/{team_id}/view",6 headers={"Authorization": f"Bearer {api_key}", "X-Team-ID": team_id},7).json()["data"]89if TeamCapability.MANAGE_BILLING.value in (view.get("can") or []):10 ...11if TeamCapability.MANAGE_ORG.value in ((view.get("org") or {}).get("can") or []):12 ...| Enum | Members (API string values) |
|---|---|
TeamCapability | edit_profile, manage_members, manage_admins, view_members, manage_keys, create_keys, manage_vault, view_billing, manage_billing, view_policy, manage_policy, manage_org, manage_sso, archive, create_team, create_org, connect_org_credential, connect_platform_credential |
Capability meanings and org vs member-workspace split are documented on Workspace settings view — Capability strings.
Problem code values — ErrorCode mirrors RFC 9457 code strings (team_role_required, payment_method_required, blocked_by_usage_policy, entitlement codes, remote harness errors, …). Use it with TeamRoleRequiredMeta and PaymentMethodRequiredMeta as in Error handling.
| Enum | Members (API string values) |
|---|---|
ErrorCode | invalid_request, validation_error, unauthorized, forbidden, not_found, team_role_required, blocked_by_usage_policy, limit_exceeded, feature_not_available, payment_method_required, remote_offline, harness_not_drivable, … — full set in inferencesh.types |
EntitlementSource | tier, override, whitelist, trial |
EntitlementResource (feature gates) | RESOURCE_FEATURE_BYOK (feature:byok), RESOURCE_FEATURE_SEEDANCE (feature:seedance), … — see Entitlements API |
WorkerStatus | reserved, busy, idle, inactive |
EngineStatus | running, pending, draining, restarting, disconnected, stopping, stopped |
InstanceRentalType | on_demand, spot — cloud catalog rows, InstanceDTO, and nested engine.instance |
Credential configs: merged setup view from GET /credentials/configs. It returns the provider catalog, connection state, and connected MCP servers. One call is enough for connect/disconnect UIs. See Credential configs.
Credential types were named
Integration*until 2026-09; import theCredential*names frominferencesh.types.
1from inferencesh.types import (2 CredentialConfigDTO,3 CredentialDTO,4 CredentialGrant,5 CredentialScope,6)78def connect_label(config: CredentialConfigDTO) -> str:9 if config.get("connection_scope") == CredentialScope.USER:10 return f"connect your {config['name']} account"11 return f"connect {config['name']}"1213def list_subtitle(credential: CredentialDTO) -> str:14 account_id = credential.get("account_identifier") or ""15 if credential.get("scope") == CredentialScope.USER:16 return f"{account_id} (your account)"17 return account_id1819per_user_oauth: CredentialConfigDTO = {20 "available": True,21 "has_managed": True,22 "allows_byok": False,23 "connection_scope": CredentialScope.USER,24 "name": "X",25}| Enum | API value | Meaning |
|---|---|---|
CredentialGrant.CREDENTIALS | credentials | OAuth application row (client id/secret and related app keys), not a login |
CredentialGrant.TOKEN | token | Connection row: OAuth login, API key, service account, or MCP authorization |
CredentialScope.USER | user | Personal connection. Use it for scope-aware list labels, section headings, and disconnect copy. |
CredentialScope.TEAM | team | Shared workspace connection |
CredentialScope.ORG | org | Shared across all workspaces of an org. Connecting one requires an org admin, from the organization workspace. |
CredentialScope.PLATFORM | platform | Platform-managed credentials |
CredentialScope.AGENT | agent | Agent-scoped credential |
inferencesh ≥ v0.7.35 models config rows with connection_scope and optional nested app (OAuth application) instead of grant on CredentialConfigDTO. grant and app_credential_id appear on CredentialDTO — use grant to tell app rows (credentials) from logins (token). scope on CredentialDTO indicates who owns the connection. CredentialConfigDTO["credential"] holds the connected login; CredentialConfigDTO["app"] holds the OAuth app when configured. See Managed credentials and grant type.
inferencesh ≥ v0.28.0 adds CredentialDTO["secrets"]: the vault keys of a key-based credential, as CredentialSecretRef (id, key, updated_at), never the values. inferencesh ≥ v0.29.0 adds terms_url and privacy_url to CompanySummary (CredentialConfigDTO["company"]), next to slug, name, icon_url, website and docs_url. CompanySummary was ProviderSummary and the field was provider_info before INF-1079. The next minor release drops provider_info and renames auth_scheme_id to auth_provider_id, matching the API.
Secrets and connection scope: use with the Secrets REST API and connection_scope on connect. The Python SDK has no client.secrets or client.credentials namespace. Import the generated types when calling these endpoints with httpx or requests:
1import httpx2from inferencesh.types import CredentialConnectRequest, CredentialScope, SecretCreateRequest34base_url = "https://api.inference.sh"5headers = {"Authorization": "Bearer inf_your_key"}67# POST /secrets: BYOK key linked to a workspace-scoped credential8byok: SecretCreateRequest = {9 "key": "OPENAI_API_KEY",10 "value": "sk-your-key-here",11 "provider": "openai",12 "connection_scope": CredentialScope.TEAM,13 "description": "Team OpenAI BYOK key",14}15httpx.post(f"{base_url}/secrets", headers=headers, json=byok)1617# POST /credentials: OAuth connect with an explicit scope18connect: CredentialConnectRequest = {19 "provider": "github",20 "type": "oauth",21 "connection_scope": CredentialScope.USER,22}23httpx.post(f"{base_url}/credentials", headers=headers, json=connect)| Enum / TypedDict | Notes |
|---|---|
CredentialScope | user, team, org, platform, agent. Who owns the credential. Setting it explicitly requires the matching admin role. |
SecretCreateRequest | Body for POST /secrets. Optional provider and connection_scope link a BYOK key to a credential at the chosen scope. |
CredentialConnectRequest | Body for POST /credentials. Optional connection_scope is separate from OAuth scopes. |
CredentialCompleteOAuthRequest | Body for POST /credentials/oauth/complete. Optional params maps extra redirect query keys (for example QuickBooks realmId, Shopify shop) for auth method templates as {{callback.*}}. inferencesh ≥ v0.11.1 adds params to the generated type (REST behavior unchanged). See Callback query parameters on OAuth complete. |
Change who owns an existing connection with PUT /credentials/{provider}/scope (body: { "scope": "team" }; MCP uses ?identifier=<server_url>). See Change connection scope.
MCP tool responses — type MCP tool proxy results (POST /mcps/{slug}/tools/{tool}) and MRTR retries with ToolCallRequest.inputResponses / requestState. inferencesh ≥ v0.13.0 adds ResultType.TASK for the MCP tasks extension and ClientCapabilities.extensions for extension negotiation during MCP initialize:
1import httpx2from inferencesh.types import ResultType, ToolCallRequest, ToolCallResponse34body: ToolCallRequest = {"name": "list_issues", "arguments": {"limit": 10}}5response: ToolCallResponse = httpx.post(6 f"{base_url}/mcps/linear/tools/list_issues",7 headers=headers,8 json=body,9).json()["data"]1011kind = response.get("resultType") or ResultType.COMPLETE.value12if kind == ResultType.INPUT_REQUIRED.value:13 retry: ToolCallRequest = {14 **body,15 "inputResponses": {...}, # keys match inputRequests from the interim response16 "requestState": response.get("requestState", ""),17 }18 response = httpx.post(19 f"{base_url}/mcps/linear/tools/list_issues",20 headers=headers,21 json=retry,22 ).json()["data"]23elif kind == ResultType.TASK.value:24 # Server accepted the call as an async MCP task; poll tasks/get for the final ToolCallResponse.25 ...| Enum / TypedDict | Notes |
|---|---|
ResultType | complete (default when omitted), input_required (MRTR), task (MCP tasks extension) |
ToolCallResponse | resultType, content, structuredContent, isError, _meta; MRTR adds inputRequests, requestState |
ToolCallRequest | name, arguments; MRTR retries add inputResponses, requestState |
ClientCapabilities | MCP client capabilities on initialize; extensions maps extension identifiers (for example tasks) to extension settings |
→ Using tools — Interactive elicitation (MRTR)
Engines — use with the Engines REST API until client.engines ships in Python (available today in the JavaScript SDK):
1from inferencesh.types import EngineDTO, EngineStatus, InstanceRentalType23engine: EngineDTO = {4 "id": "eng_abc123",5 "name": "gpu-box-1",6 "status": EngineStatus.RUNNING,7 "engine_version": "v1.10.15", # inferencesh-engine binary; empty until first telemetry sync8 "api_url": "https://api.inference.sh",9 "workers": [],10 "instance": {11 "rental_type": InstanceRentalType.SPOT,12 "hourly_price": 199,13 },14}Spot cloud engines stop permanently when the provider reclaims the instance; provision a new engine or use on-demand — see Engines API — Get engine.
Pages: use with the Pages REST API and workspace username checks until dedicated client namespaces ship:
1from inferencesh.types import (2 AvailabilityResponse,3 PageDTO,4 PageMetadata,5 PageStatus,6 PageType,7)89# GET /pages/slug/{slug}10page: PageDTO = {11 "title": "Scheduled launch post",12 "slug": "launch-day",13 "status": PageStatus.SCHEDULED,14 "type": PageType.BLOG,15 "metadata": {16 "publish_at": "2026-08-01T09:00:00Z", # write here on create/update17 },18 "publish_at": "2026-08-01T09:00:00Z", # mirrored on read — do not require metadata drill-down19}2021slug_check: AvailabilityResponse = {22 "value": "my-new-post", # normalized slug from the server23 "available": False,24 "reason": "taken", # or "reserved" — only set when available is false25}| Enum / TypedDict | Notes |
|---|---|
PageStatus | UNKNOWN (0), DRAFT (1), PUBLISHED (2), ARCHIVED (3), SCHEDULED (4) |
PageType | doc, blog, page |
PageDTO | Includes top-level publish_at (RFC 3339) alongside metadata.publish_at |
AvailabilityResponse | Shared by page slug and workspace username availability endpoints. Replaces the older SlugAvailabilityResponse shape (slug → value, plus optional reason) |
Interrupt gates
Use with the Agents REST API — Interrupt gates (GET /agent-runs/{runId}/interrupts, POST /interrupts/{id}/resolve). The JavaScript SDK exposes client.agents.listRunInterrupts() and resolveInterrupt(); in Python, call the REST endpoints directly and type responses with inferencesh.types (v0.7.23+).
1from inferencesh.types import (2 AgentRunDTO,3 InterruptDTO,4 InterruptReason,5 InterruptStatus,6 InterruptResolution,7 InterruptResourceType,8)910# active_run on GET /chats/:id when the run is paused11run: AgentRunDTO = chat["active_run"]12if run and run.get("interrupt_reason") == InterruptReason.HOOK_GATE:13 ...1415# GET /agent-runs/{runId}/interrupts16pending: list[InterruptDTO] = [17 {18 "id": "int_abc123",19 "run_id": "ar_abc123",20 "chat_id": "chat_abc123",21 "reason": InterruptReason.TOOL_APPROVAL,22 "status": InterruptStatus.PENDING,23 "resource_id": "ti_abc123",24 "resource_type": InterruptResourceType.INTERRUPT_RESOURCE_TOOL_INVOCATION,25 }26]2728# POST /interrupts/{id}/resolve response29resolved: InterruptDTO = {30 "id": "int_abc123",31 "status": InterruptStatus.RESOLVED,32 "resolution": InterruptResolution.ALLOW,33 "resolved_at": "2026-08-11T15:00:00Z",34 "resolved_data": {"reviewer": "security-bot"},35}| Enum / TypedDict | Notes |
|---|---|
InterruptReason | tool_approval, client_tool, widget, auth, confirmation, hook_gate — mirrors active_run.interrupt_reason |
InterruptStatus | pending, resolved, expired, cancelled |
InterruptResolution | allow, deny — set on InterruptDTO after resolve; also used as default_resolution on gate hooks |
InterruptResourceType | tool_invocation (tool approval gates), hook_event (lifecycle hook gates) |
InterruptDTO | Full interrupt record (run_id, chat_id, source, resource_id, resource_type, meta, resolved_data, expires_at, …) |
AgentRunDTO also includes interrupt_reason, interrupt_tool_id, and interrupt_meta when the run is in input_required or auth_required state.
Remote placement: when an agent runs on a remote harness profile, typed REST responses include optional profile_id and remote_id on AgentRunDTO and AgentDTO, plus harness_session_id and forked_from_message_id on ChatDTO. See Agents REST API — Remote placement and Remote harness runs.
Remote harness chats
inferencesh ≥ v0.10.3 exports OpenAPI fields for harness-driven agents and chats opened via Launch harness session. There is no client.remotes namespace yet; call POST /remotes/{id}/sessions with httpx or requests and type responses with ChatDTO and AgentDTO from inferencesh.types.
1import httpx2from typing import TypedDict, NotRequired34from inferencesh.types import AgentDTO, ChatDTO56API = "https://api.inference.sh"7headers = {"Authorization": "Bearer inf_your_key"}89agent: AgentDTO = httpx.get(f"{API}/agents/agent_abc123", headers=headers).json()["data"]10if agent.get("harness") != "inference":11 print(agent["harness"], agent.get("profile_id"), agent.get("remote_id"))1213chat: ChatDTO = httpx.get(f"{API}/chats/chat_abc123", headers=headers).json()["data"]14print(chat.get("work_dir"), chat.get("harness_session_id"))1516class LaunchHarnessSessionRequest(TypedDict):17 profile_id: str18 work_dir: NotRequired[str]19 resume_session_id: NotRequired[str]2021launch_body: LaunchHarnessSessionRequest = {22 "profile_id": "prof_xyz",23 "work_dir": "/home/you/project",24}2526launched: ChatDTO = httpx.post(27 f"{API}/remotes/remote_abc123/sessions",28 headers=headers,29 json=launch_body,30).json()["data"]| Field / type | Description |
|---|---|
AgentDTO.harness | "inference" for the platform loop, or a harness registry id (claude, codex, …) when a remote profile drives the agent |
AgentDTO.profile_id / AgentDTO.remote_id | Set on harness-backed agents (see REST docs) |
ChatDTO.work_dir | Folder the harness uses for this chat |
ChatDTO.harness_session_id | Harness-native session id (for example to resume with claude --resume) |
ChatDTO.channel_context | When set, external channel origin (channel_type, channel_metadata) for reply routing — see Channel context |
App requirement and setup-action types
Use with Check requirements and the credentials: / secrets: blocks of inf.yml:
1from inferencesh.types import (2 CheckRequirementsRequest,3 CredentialRequirement,4 SecretRequirement,5 SetupAction,6 SetupActionType,7)89req: CheckRequirementsRequest = {10 "secrets": [{"key": "OPENAI_API_KEY", "optional": False}],11 "credentials": [12 {"slug": "google", "capabilities": ["google.sheets"]}, # platform capability13 {"slug": "acme", "secrets": ["ACME_API_KEY"], # provider keys14 "provider": {"name": "Acme CRM", "website": "acme.com"}},15 ],16}inferencesh ≥ v0.26.0 types CredentialRequirement by slug and capabilities, matching the slug-keyed credentials: in inf.yml. The API still reads the older list spellings (key for a capability, provider as a slug string, name / website on the entry).
| Type | Fields |
|---|---|
SecretRequirement | key, description, optional |
CredentialRequirement | slug (credential slug such as google or acme; derived from a capability-only entry), capabilities (platform grants such as google.sheets), secrets (keys to inject from the credential), scopes (OAuth scopes to request), description, optional, provider (CompanySeed, for a company the platform does not list; company: in inf.yml), auth (AuthMethod, for a slug no auth provider defines yet). CompanySeed and AuthMethod were ProviderSeed and AuthSchemeSpec before INF-1079; AuthSchemeDTO is now AuthProviderDTO |
CompanySeed | slug (defaults to the credential slug), name, description, website, logo_uri, docs_uri, tos_uri, policy_uri |
SetupAction | type (SetupActionType: add_secret, connect, add_scopes), provider, provider_name, scopes, scope_descriptions, secrets (keys to supply for an add_secret action), provider_website |
CheckRequirementsResponse | satisfied, errors (list of RequirementError, each carrying an optional action: SetupAction) |
Lifecycle hook types
Use with Lifecycle hooks and GET /agents/hook-events:
1from inferencesh.types import (2 BuiltinHook,3 HookEvent,4 HookHandlerType,5 HookDecision,6 HookEventDefinition,7 InterruptResolution,8 LifecycleHookConfig,9)1011# GET /agents/hook-events12events: list[HookEventDefinition] = [13 {14 "event": HookEvent.TOOL_CALL,15 "description": "Fires before a tool is executed",16 "can_gate": True,17 }18]1920# Gate hook on agent.tool_call (no builder helper yet — set type directly)21gate_hook: LifecycleHookConfig = {22 "event": HookEvent.TOOL_CALL,23 "type": HookHandlerType.HOOK_HANDLER_GATE,24 "timeout": 600,25 "default_resolution": InterruptResolution.DENY,26}2728# Builtin belt hooks (inferencesh >= v0.14.2)29from inferencesh import learning_hooks, lifecycle_hook3031belt_hooks = learning_hooks(suggest=True, learn=True)32single = lifecycle_hook(HookEvent.TURN_START).builtin(BuiltinHook.BELT_SUGGEST).build()| Enum / TypedDict | Notes |
|---|---|
HookEvent | agent.start, agent.turn_start, agent.tool_call, agent.tool_result, … |
HookHandlerType | webhook, task, gate, builtin |
BuiltinHook | belt:suggest (agent.turn_start), belt:extract (agent.complete, agent.pre_compact); see Lifecycle hooks — Builtin |
HookDecision | allow, deny, stop, suspend — suspend is returned by gate handlers |
HookEventDefinition | Response shape from GET /agents/hook-events (event, description, can_gate) |
BuiltinHookDefinition | Builtin hook catalog entry (name, description, allowed events) — inferencesh ≥ v0.13.0 |
LifecycleHookConfig | Agent version hooks[] entry; gate hooks add default_resolution |
ContextInjection.items | Resource refs or paths a hook injection placed in context (dedup across turns) — inferencesh ≥ v0.13.0 |
Flow utility nodes
Use with the Flows REST API when changing flow graphs with httpx or requests.
1from inferencesh.types import FlowNodeData23# Preferred: unified utility config on node_data4gate_data: FlowNodeData = {5 "utility": {6 "preset": "gate",7 "gate": {"field": "status", "operator": "equals", "value": "ready"},8 },9}1011selector_data: FlowNodeData = {12 "utility": {13 "preset": "selector",14 "selector": {"field": "items", "mode": "first"},15 },16}1718constant_data: FlowNodeData = {19 "utility": {"preset": "constant", "constant": 42},20}2122# Legacy fields, used when "utility" is not set23legacy_gate: FlowNodeData = {24 "gate_condition": {"field": "status", "operator": "equals", "value": "ready"},25}26legacy_selector: FlowNodeData = {27 "selector_config": {"field": "items", "mode": "index", "index": 0},28}| TypedDict | Fields |
|---|---|
FlowNodeData | Graph node payload. Has utility plus the legacy gate_condition and selector_config |
UtilityConfig | preset (gate, selector, merge, constant), expression, gate, selector, constant, random, random_min, random_max |
GateCondition | field, operator (equals, not_equals, contains, gt, lt, is_empty, is_not_empty), value |
SelectorConfig | field, mode (first, last, index), index (when mode is index) |
Task files — use with List task files and Delete task files. inferencesh ≥ v0.19.0 ships TaskFileDTO, DeleteTaskFilesResponse, and TaskFileRole alongside client.tasks.files() / delete_files():
1from inferencesh.types import DeleteTaskFilesResponse, TaskFileDTO, TaskFileRole23file_row: TaskFileDTO = {4 "id": "file_abc123",5 "role": TaskFileRole.OUTPUT,6 "uri": "https://cloud.inference.sh/t/abc123/image.png",7 "filename": "image.png",8 "content_type": "image/png",9 "size": 1048576,10}1112purge: DeleteTaskFilesResponse = {13 "deleted": ["file_abc123"],14 "skipped": [{"id": "file_other", "reason": "forbidden"}],15}| TypedDict / enum | Notes |
|---|---|
TaskFileDTO | One file from GET /tasks/{id}/files |
DeleteTaskFilesResponse | Body from DELETE /tasks/{id}/files |
TaskFileRole | input, output — optional role filter on list/delete |
Flow run progress
Use with Flow Runs REST API: Per-node progress. The Python SDK has no flow runs client yet. Call GET /flowruns/{id} and type the response:
1import httpx2from inferencesh.types import FlowRunDTO, GraphNodeStatus34headers = {"Authorization": "Bearer inf_your_key"}5run: FlowRunDTO = httpx.get(6 "https://api.inference.sh/flowruns/flowrun_abc123", headers=headers7).json()["data"]89statuses = run.get("node_statuses") or {}1011# A gate whose condition is false is reported as failed; its dependents are blocked12if statuses.get("gate_1") == GraphNodeStatus.FAILED:13 print("Gate condition was false")1415if statuses.get("upscale") == GraphNodeStatus.BLOCKED:16 print("Upscale did not run")1718if statuses.get("generate") == GraphNodeStatus.RUNNING:19 print("Still running:", run["node_tasks"]["generate"]["task_id"])2021outputs = run.get("node_outputs") or {}22print(outputs.get("selector_1", {}).get("value"))| Type / enum | Description |
|---|---|
FlowRunDTO | Flow run response. Includes node_tasks, node_statuses, and node_outputs |
GraphNodeStatus | PENDING, READY, RUNNING, COMPLETED, FAILED, CANCELLED, SKIPPED, BLOCKED |
File Upload
1# Upload separately2file = client.files.upload("/path/to/image.png")3print(f"Uploaded: {file['uri']}")45# Use in app input6result = client.tasks.run({7 "app": "image-processor",8 "input": {"image": file["uri"]}9})1011# Or auto-upload from paths in task input12result = client.tasks.run({13 "app": "image-processor",14 "input": {"image": "/path/to/local/image.png"} # Auto-uploaded15})REST API (curl)
For simple integrations, you can use the REST API directly with streaming:
1# Single call with streaming2curl -N -X POST \3 -H "Authorization: Bearer YOUR_API_KEY" \4 -H "Content-Type: application/json" \5 -d '{6 "agent": "my-org/assistant@abc123",7 "input": {"text": "Hello!", "role": "user"},8 "stream": true9 }' \10 https://api.inference.sh/agents/runThe -N flag disables curl's buffering so you see events in real-time.
Without streaming (returns JSON with chat_id for subsequent messages):
1curl -X POST \2 -H "Authorization: Bearer YOUR_API_KEY" \3 -H "Content-Type: application/json" \4 -d '{5 "agent": "my-org/assistant@abc123",6 "input": {"text": "Hello!", "role": "user"}7 }' \8 https://api.inference.sh/agents/runWhat's Next?
- JavaScript SDK — JS/TS SDK reference
- Tool Builder — Full tool builder API
- REST API — Direct HTTP endpoints