Managing Engines

Drain, update, and restart engines without interrupting in-flight tasks.

Provision a cloud engine

Rent a GPU VM managed by inference.sh (Shadeform) when you want dedicated private workers without installing the engine on your own hardware.

1. Open Resources → Engines (/engines) and click cloud engine (or open /engines#create-remote).
2. Select GPU configuration — pick a GPU type and size. Each card shows the lowest in-stock hourly rate for that shape (from $…/hr when more than one cloud provider can supply it), how many providers are available, and an estimated monthly cost. Offers include on-demand and spot rental types when stock exists.
3. Configure your engine — choose the provider and region to launch (the cheapest provider is selected by default). The confirm step lists in-stock providers for that offer (first four, with show all when there are more). When a provider serves multiple regions, pick a region from the dropdown; per-region hourly prices appear when they differ (common for spot). Pick a duration between 1 hour and 30 days, set an optional spend cap, name the engine, and confirm. The form keeps that duration and turns it into end_time only when you submit, so time on the confirm step does not shorten the rental. Billing starts when the API receives the launch request and runs through that end_time; the total price uses the selected provider and region.

The workspace calls POST /engines with end_time plus cloud, shade_instance_type, region, and rental_type from your selection. Catalog data comes from GET /engines/types.

If provisioning fails after you confirm, you are not left with a half-created engine: the API refunds the reservation debit and removes any cloud instance or API key from the failed attempt. See Create cloud engine — Errors and billing.

After the instance boots and registers, the engine behaves like a self-hosted engine for drain, Update, and task routing with infra: "private".

When to use drain vs update

Both drain and update require the engine to be connected to the API over WebSocket. If the engine is offline, the API returns an error.

In the workspace

In the sidebar, open Resources → Engines (/engines), select an engine, and use:

• version (stats bar) — running inferencesh-engine release when the engine is connected; also listed under system info on the detail page.
• Drain — marks the engine as draining, rejects new tasks with engine_draining, and shuts down once all workers are idle.
• Update — triggers a safe remote restart (check for newer binary → drain → restart).

While draining, status shows draining. Queued tasks that target this engine may wait until another worker is available.

On the server (local CLI)

For engines you manage directly on Linux, use the engine CLI (alias for inferencesh-engine):

1# Check for updates and install the latest release2engine update34# Install a specific version5engine update --version v1.10.1567# Print current version8engine version

The engine also checks for updates on startup (rate-limited to once every five minutes). Pass --disable-auto-update to skip the startup check for a single run.

After a config change in engine.yml, restart the engine process (for example sudo systemctl restart inferencesh-engine if you use the example unit).

A config-only restart does not run the remote Update flow; use engine update or the workspace Update button when you need a new engine binary.

Via API and SDK

Remote drain and update are available over the REST API and the JavaScript SDK (client.engines).

1# Drain (engine must be online)2curl -X POST "https://api.inference.sh/engines/{engine_id}/drain" \3  -H "Authorization: Bearer inf_your_key"45# Update binary (drain + restart with latest binary)6curl -X POST "https://api.inference.sh/engines/{engine_id}/update" \7  -H "Authorization: Bearer inf_your_key"

1await client.engines.drain(engineId);2await client.engines.updateBinary(engineId);

Requires API key scopes engines:read (list/get) and engines:write (drain, update, stop).

→ Engines REST API · Configuration · Using private workers

What happens during drain

1. The API sets engine status to draining and sends an engine_drain event to the connected engine.
2. The engine stops accepting new tasks (engine_draining rejection reason).
3. In-flight tasks on each worker run to completion.
4. When all workers are idle, the engine shuts down gracefully.

Press Ctrl+C twice on the server to force shutdown while draining.

What happens during update

1. The API sends an engine_update event to the connected engine.
2. The engine checks the release manifest. When a newer binary exists, it downloads and swaps it in place; if already on the latest version (or the check fails), it keeps the current binary.
3. The engine enters the same drain flow as above, then restarts (process re-exec) after workers finish.

If the engine is not connected, update and drain requests fail with an internal error from the API.

Next

→ Back to Docs