Server Proxy

Protect your API key in frontend applications.

When building client-side applications (Next.js, React, Vue, etc.), you shouldn't expose your API key in browser code. The server proxy routes requests through your backend, keeping credentials safe.

How It Works

1Browser (SDK Client)  →  Your Server (Proxy)  →  api.inference.sh2     ↓                         ↓                        ↓3  proxyUrl config        Injects API key         Processes request

1. Client sends request to your proxy endpoint (e.g., /api/inference/proxy)
2. Proxy reads INFERENCE_API_KEY from environment
3. Proxy forwards request to inference shell with authentication
4. Response passes back through to client

Choose Your Framework

Configure the Client

Once you've set up a proxy route, configure the SDK client:

1import { inference } from "@inferencesh/sdk";23const client = inference({4  proxyUrl: "/api/inference/proxy"  // Your proxy route5});67const result = await client.run({8  app: "infsh/flux",9  input: { prompt: "A sunset over mountains" }10});

Note: When using proxyUrl, you don't need to provide apiKey - the proxy handles authentication.

What the Proxy Forwards

The proxy forwards only what a frontend needs to run things:

• Apps: POST /apps/run, file uploads for inputs (POST /files), and reading, polling, streaming and cancelling the task.
• Agents: POST /agents/run, POST /chats, messages in a chat, handing a chat to another agent, an agent's info (GET /agents/{namespace}/{name}), and reading, polling, streaming and stopping the chat.
• Inside a chat: results of client tools (POST /tools/{id}).

Everything else (secrets, API keys, billing, your account, task and chat listings, file listings, creating or changing apps and agents, chat settings, admin) is refused with 403 before the API key is attached. No option opens these.

POST /apps/run takes only app (or app_id / version_id), function, input, stream and wait. A visitor cannot set workers, infra, session, session_timeout, webhook, run_at, setup or metadata.

Target host

The proxy attaches the key only to requests for the API your frontend's client calls: apiUrl (default https://api.inference.sh), or apiBaseUrl / INFERENCE_API_BASE_URL when set, which also rewrites every target to that base. Other hosts, other inference.sh hosts included, are refused with 412. allowedDomains adds hosts; each pattern must match the whole host. Targets must be https, except a loopback host in development.

Tool approvals don't work through a proxy key

The API takes an approval (approve, reject, always allow, resolving an interrupt, answering a question the agent asked the person) only from the account holder's own sign-in, never from an API key. The proxy signs every request with your key, so it does not forward these calls, nor "allow all tools".

For a public site, use an agent whose tools don't need approval, or publish the agent and embed it: embed visitors can approve tool calls in their own chat.

Anyone with a task or chat id can use it

All visitors share your API key, so every task and chat started through the proxy belongs to the same account. The proxy can't tell visitors apart. Whoever holds a task or chat id can read and stream it, cancel or stop it, and send messages in the chat, which run the chat's agent whatever allowedEndpoints says. Ids are random and hard to guess, but the id is the only boundary between visitors. Don't show or log one visitor's ids to another. If visitors must not reach each other's chats, check ownership in isAuthenticated (map ids to your own sessions), or give each tenant its own key.

Uploads

POST /files takes one file per request, with its declared size up to maxUploadBytes (default 100 MiB). The browser then uploads the bytes straight to storage. Uploaded files count against the storage quota of the account that owns the API key. Rate-limit the route.

Options

Every framework handler takes these options:

1import { createHandler } from "@inferencesh/sdk/proxy/nextjs";23export const { GET, POST } = createHandler({4  allowedEndpoints: ["my-team/support-agent", "infsh/flux"],5  isAuthenticated: async (req) => Boolean(await getSession(req)),6});

allowedEndpoints is checked where a request names an app or agent: a new run, a new chat or message, a chat handed to another agent, an agent's info. A request about an existing task or chat names it by id, and the proxy can't tell which app or agent that is: see anyone with a task or chat id can use it. With allowedEndpoints set, ad-hoc agent configs (agent_config) and runs by app_id are refused.

[!WARNING] Production. Without isAuthenticated, anyone who can reach your proxy route runs apps and agents on your API key. Before going live:

• set allowedEndpoints to the apps and agents your frontend uses
• add isAuthenticated, or set allowUnauthorizedRequests: false if only signed-in users may call it
• create a Run only key (Settings → API keys → permissions → run only) in a workspace that holds nothing else. It holds exactly what the proxy forwards: apps:read, apps:execute, agents:read, agents:execute, conversations:read, conversations:write and files:write
• rate-limit the route, uploads included: they use your storage quota
• serve agents whose tools don't need approval (approvals don't work through a key)

[!IMPORTANT] Never expose your API key in client-side code. The proxy exists specifically to keep credentials on your server.

Authentication precedence

When forwarding to api.inference.sh, the proxy resolves credentials in this order:

1. Client Authorization header — forwarded as-is when present (for example a session token from device authorization)
2. options.apiKey, or resolveApiKey on the SvelteKit, Hono and Remix handlers
3. INFERENCE_API_KEY environment variable — injected as Bearer … when no client header is sent

This lets browser clients pass short-lived session tokens through your proxy without storing a long-lived API key on the server.

Forwarded request headers

The proxy forwards these to the upstream request:

• x-inf-* — any header with this prefix (for example x-inf-session-id)
• X-Client-Source — SDK name and version, used for telemetry attribution

Custom Proxy Logic

Custom API Key Resolution

The SvelteKit, Hono and Remix handlers take resolveApiKey, called on each request:

1// Load from secrets manager instead of env2import { createHandler } from "@inferencesh/sdk/proxy/svelte";34const handler = createHandler({5  resolveApiKey: async () => {6    return await vault.getSecret("INFERENCE_API_KEY");7  }8});

The Next.js and Express handlers take only apiKey, a string. Load the secret before you create the handler:

1import { createHandler } from "@inferencesh/sdk/proxy/nextjs";23const apiKey = await vault.getSecret("INFERENCE_API_KEY");45export const { GET, POST } = createHandler({ apiKey, allowedEndpoints: ["my-team/*"] });

Next Steps

• Vercel Deployment — Deploy with automatic env setup
• Running Apps — Execute AI tasks
• Streaming — Real-time progress updates