Engines

Manage private inference engines that run workers on your hardware. Use these endpoints to list engines, inspect status, and perform safe restarts without dropping in-flight tasks.

Requires API key scopes engines:read (list/get) and engines:write (drain, update, stop, restart).

→ Workers · Installing the engine · Configuration

Register engine

POST /engines/register

Called by the inferencesh-engine binary on startup (after registration). Creates a new engine record for your workspace or re-registers an existing one, allocates workers from detected hardware, and returns worker IDs the engine uses for task routing.

Requires engines:write. You normally do not call this manually — install the engine and run engine serve instead.

Request

Response

On first run, the API creates a new engine with status pending. On subsequent starts, the engine sends its saved ID; the API updates hardware info and worker allocation, sets status to pending, and clears any prior stop reason (starting the process is an explicit start, including after you Stop an engine in the workspace).

Register does not set running. The engine opens its WebSocket after register; that connection moves status to running. Between register and connect you may briefly see pending in list or get responses — nothing in the API waits on running during that gap.

After connect, the engine maintains the WebSocket for task dispatch, drain/update signals, and heartbeats. The API treats that connection as the source of truth for whether the engine is online. See Installing the engine for setup steps.

List engines

GET /engines or POST /engines/list

Cursor-paginated list of engines for your workspace.

1curl -X POST https://api.inference.sh/engines/list \2  -H "Authorization: Bearer inf_your_key" \3  -H "Content-Type: application/json" \4  -d '{"limit": 20}'

Get engine

GET /engines/{id}

Returns an engine DTO. List responses (POST /engines/list) use the same shape per item.

While connected, heartbeats report the finer states above (running, draining, or restarting). When the WebSocket drops, status moves to disconnected even if the host process is still running. A new connection moves the engine back to running from disconnected, pending, or restarting, and can revive a stopped engine after a disconnect-only stop. A Stop you issued or a completed drain is final for reconnect alone — run the engine again (engine serve) so register clears the stop and sets pending, then connect sets running.

Spot cloud engines: when the cloud provider reclaims a spot instance (instance.rental_type is spot), the engine is finalized as stopped with stop_reason: "preempted" and does not come back on reconnect. Unused prepaid time is refunded as for a stop. Provision a new cloud engine (or switch to on-demand) to replace the capacity.

If an engine stays disconnected for about five minutes, the API finalizes it: workers are marked inactive, in-flight tasks fail, and cloud instances may be torn down. Brief outages (for example during an API deployment) normally recover when the engine reconnects before that window.

While the engine is still in that reconnect grace (disconnected, not yet finalized), worker liveness follows the engine, not a separate wall-clock on each worker heartbeat. A WebSocket drop does not, by itself, retire workers or fail their in-flight tasks — work can keep running on the host and resume dispatch when the connection returns. Workers are retired when the engine is finalized, when you stop or delete the engine, or when the engine stays connected but has not reported a given worker in its heartbeats for about 90 seconds (for example, a crashed worker slot). API downtime does not count against workers either: engine heartbeats stop being recorded at the same time, so workers are not marked gone during a platform restart.

If the platform ends a task while your engine could not be reached (for example a Dispatched task that fails after about 30 seconds without a connection), the task row is already Failed or Cancelled but a worker may still be running that work on the host. On the next engine telemetry report after reconnect (on connect and about every 30 seconds while connected), the API compares what each worker reports against the task record and sends a force cancel to any worker still running a task the platform no longer owns on that engine. That stops stray side effects without re-queueing the same run on another engine. Completed tasks are never treated as stale — a worker may still be uploading results after the compute phase finishes.

JavaScript SDK: EngineStatusDisconnected, EngineStatusRestarting, and other status constants ship in @inferencesh/sdk ≥ v0.6.12. EngineDTO includes engine_version, matching the REST API response. Python SDK: EngineDTO in inferencesh.types includes the same field — use it when calling GET /engines/{id} directly.

1curl "https://api.inference.sh/engines/eng_abc123" \2  -H "Authorization: Bearer inf_your_key"

Match engines to resource requirements

POST /engines/resources

Check which workers can satisfy a given resource profile before you run an app with infra: "private" or choose between cloud and private capacity. The web app's run form uses this endpoint to show available private workers and cloud price ranges.

Authentication is optional (guest workspaces get public cloud availability only). With an API key, your workspace's private workers are included in the response.

Request

Pass the resources block from the app version you plan to run (see App configuration). Values are in bytes — the same internal representation stored on deployed app versions after CLI deploy.

1curl -X POST https://api.inference.sh/engines/resources \2  -H "Authorization: Bearer inf_your_key" \3  -H "Content-Type: application/json" \4  -d '{5    "resources": {6      "gpu": { "count": 1, "vram": 24000000000, "type": "nvidia" },7      "ram": 160000000008    }9  }'

Response

Each worker summary includes id, engine_id, engine_name, status, GPU/CPU/RAM details, and the current task_id when busy.

Use this before POST /run when you need to confirm capacity or pick specific workers — see Using private workers.

→ Instance types · Workers concept

List engine instance types

GET /engines/types

Authenticated catalog for provisioning cloud engines in the workspace. Requires engines:read.

Returns one row per GPU type, GPU count, and rental type (on_demand or spot): the cheapest available option for that tuple, or for on-demand the cheapest unavailable row when nothing is in stock. Spot offers are omitted when no region has spot stock.

Each row is an instance type object with rental_type, region, and hourly_price set for that offer (the raw public catalog leaves rental_type empty). Spot row IDs are suffixed with .spot.

Each offer also includes an options array: every in-stock cloud provider for that GPU type, GPU count, and rental type. Options are sorted by cheapest hourly_price first; the first entry always matches the top-level offer (cloud, shade_instance_type, region, and hourly_price). Providers with no in-stock regions for that rental type are omitted.

Use the parent row's rental_type when launching. Pick cloud, shade_instance_type, and region from whichever options[] entry the user selects (not only the top-level offer fields). In the web app, the cloud engine confirm step exposes the same choice as a provider list and region selector — see Provision a cloud engine.

1curl "https://api.inference.sh/engines/types" \2  -H "Authorization: Bearer inf_your_key"

Example fragment (one on-demand H100 offer with two providers):

1{2  "id": "runpod.NVIDIA H100 80GB HBM3",3  "cloud": "runpod",4  "region": "eu-north-1",5  "shade_instance_type": "NVIDIA H100 80GB HBM3",6  "rental_type": "on_demand",7  "hourly_price": 250,8  "options": [9    {10      "cloud": "runpod",11      "shade_instance_type": "NVIDIA H100 80GB HBM3",12      "hourly_price": 250,13      "regions": [{ "region": "eu-north-1", "hourly_price": 250 }]14    },15    {16      "cloud": "lambdalabs",17      "shade_instance_type": "gpu-h100-sxm-80gb",18      "hourly_price": 300,19      "regions": [20        { "region": "us-east-1", "hourly_price": 300 },21        { "region": "us-west-2", "hourly_price": 300 }22      ]23    }24  ]25}

→ Instance types API for the full public GET /instances/types catalog and filters.

Create cloud engine

POST /engines

Provisions a new cloud engine on Shadeform. Requires engines:write. The rental starts when the API receives the request and runs until end_time, which is also the cloud instance auto-delete threshold. Your workspace balance is debited in microcents for that span (from now through end_time) using the server-quoted hourly rate — the hourly_price field on instance_type is ignored.

end_time must be at least 1 hour and at most 30 days ahead of the request. The API allows about one minute of slack on the minimum so a client-computed end_time is not rejected by network latency. Requests outside that window return 400 with code invalid_request.

There is no start_time field — billing always begins at request arrival. When you expose a duration picker in a UI, compute end_time at submit from the user's chosen span (not when they opened the form) so the prepaid window matches what they selected. The inference.sh workspace does this for cloud engine launch.

Request

1curl -X POST https://api.inference.sh/engines \2  -H "Authorization: Bearer inf_your_key" \3  -H "Content-Type: application/json" \4  -d '{5    "name": "h100-spot-1",6    "end_time": "2026-10-01T20:00:00Z",7    "instance_type": {8      "cloud": "runpod",9      "region": "us-east-1",10      "shade_instance_type": "NVIDIA H100 80GB HBM3",11      "rental_type": "spot"12    }13  }'

The response is the created engine record (including instance with rental_type and quoted hourly_price). The engine boots via a launch script and moves to running after register and WebSocket connect — same lifecycle as a self-hosted engine after that point.

Spot instances cost less but can be reclaimed by the provider at any time; see Get engine for behavior when that happens.

Errors and billing

If Shadeform cannot quote the instance (for example the offer is unavailable), the request fails before your balance is debited. Failures other than an invalid end_time return 500 internal_error with a generic message.

After the reservation debit succeeds, the API provisions the cloud instance and creates the engine record. If any later launch step fails, the request returns an error and no engine is left in your workspace — the API deletes the engine row if one was created, reverses the reservation debit, removes any engine API key from the attempt, and tears down any cloud instance that was started. The unused-time credit from Stop only applies when a cloud engine finishes normally.

Check your workspace billing history if a launch failed but you expected a charge or refund line item.

Balance charges: Launch, extend, and stop refunds all move workspace balance in microcents (see Billing API — Money units). Catalog and instance hourly_price values are cents per hour; runtime cost prorates by elapsed seconds at that rate (for example, three hours at 138 cents/hour is 414 cents of usage, stored as microcents on the balance). Each extend call debits separately; when you stop a cloud engine early, the API credits unused prepaid time (launch plus extensions) minus elapsed usage since the engine was created. In billing history, look for references such as cloud_engine:<name> (launch debit), cloud_engine_extend:<engine id> (extension debits), and engine:<engine id> stop refund (early-stop credit).

Drain engine

POST /engines/{id}/drain

Gracefully prepare an engine for shutdown:

1. Marks the engine as draining in the API.
2. Sends a drain signal over the engine WebSocket.
3. The engine stops accepting new tasks and waits for in-flight work to finish.
4. When all workers are idle, the engine shuts down.

The engine must be connected (online WebSocket). If it is offline, the API returns an error.

Use drain before maintenance or when you want a clean shutdown without cutting off running tasks.

1curl -X POST "https://api.inference.sh/engines/eng_abc123/drain" \2  -H "Authorization: Bearer inf_your_key"

While draining, new tasks routed to that engine are rejected with reason engine_draining.

Update engine binary

POST /engines/{id}/update

Triggers a safe engine restart on a connected private engine:

1. The API sends an update signal over the engine WebSocket.
2. The engine checks https://dist.inference.sh/engine/manifest.json for a newer binary (same OS/arch). If one exists, it downloads and swaps the binary before restarting.
3. The engine drains in-flight tasks and restarts the process (re-exec) after workers are idle — even when already on the latest version.

This is the API equivalent of the web app's Update action on Engines or running engine update on the host. Use it for binary upgrades or a clean restart without killing running tasks.

1curl -X POST "https://api.inference.sh/engines/eng_abc123/update" \2  -H "Authorization: Bearer inf_your_key"

The engine must be online. On hosts you manage directly, you can also update with the engine CLI — see Installing the engine.

Extend cloud engine duration

POST /engines/{id}/extend

Adds runtime to a cloud-provisioned engine (Shadeform instance) before its auto-delete threshold. Requires engines:write. Your workspace balance is debited in microcents at hourly_price × additional_hours (with hourly_price in cents per hour).

Request

The engine must have an active instance with auto_delete.date_threshold and a positive hourly_price synced from the provider.

1curl -X POST "https://api.inference.sh/engines/eng_abc123/extend" \2  -H "Authorization: Bearer inf_your_key" \3  -H "Content-Type: application/json" \4  -d '{"additional_hours": 3}'

Returns the updated engine DTO with a new instance auto_delete threshold. The new threshold cannot be more than 30 days ahead of the request (same cap as create); otherwise the API returns 400 with code extend_failed.

Other errors use 400 with code extend_failed when the instance cannot be extended (for example, missing threshold or price).

Self-hosted engines on your own hardware do not use this endpoint — use Update engine binary or restart the service on the host.

Stop engine

POST /engines/{id}/stop

Stops a managed engine instance. If the engine is connected, it receives a graceful stop signal, finishes in-flight tasks, then terminates (including cloud instance teardown when applicable). If the engine is offline, the API finalizes the stop immediately.

For cloud-provisioned engines with a prepaid reservation, finalize also credits unused prepaid time minus elapsed usage (see Create cloud engine — Balance charges). Self-hosted engines on your own hardware are not billed through this path.

Restart engine

POST /engines/{id}/restart

Restarts the underlying cloud instance (Shadeform) for engines provisioned through inference.sh. For self-hosted engines on your own hardware, use update or restart the inferencesh-engine process on the host.

JavaScript SDK

1import { inference } from '@inferencesh/sdk';23const client = inference({ apiKey: 'inf_your_key' });45await client.engines.drain('eng_abc123');6await client.engines.updateBinary('eng_abc123'); // POST /engines/{id}/update7await client.engines.stop('eng_abc123');8await client.engines.restart('eng_abc123');9// POST /engines/{id}/extend — pass { additional_hours } in the request body (REST or HttpClient)

client.engines also exposes list, get, create, update, delete, and stream.

Related