# Workers

> Workers are computers that run your tasks.

**URL:** https://inference.sh/docs/concepts/workers
**Last updated:** 2026-10-12

---

Workers are computers that run your tasks.

---

## Two types

| Type | Description |
|------|-------------|
| **Cloud** | Managed by inference shell |
| **Private** | Your own hardware |

---

## Cloud workers

- Pay-per-use
- Auto-scaling
- No setup required

Good for getting started and variable workloads.

For programmatic GPU pricing and availability across providers, use the public [Instance types API](/docs/api/rest/instances) (`GET /instances/types`).

---

## Private workers

- Run on your hardware
- Via the inference shell **Engine**
- Data stays on your network

Good for data privacy, dedicated resources, or cost control.

→ [Engines API](/docs/api/rest/engines) — drain, update, and lifecycle for private engines

---

## How tasks find workers

1. You run an app
2. Task goes to the queue
3. A worker picks it up when one is available (tasks wait in **Queued** if capacity is temporarily full)
4. Worker runs it and returns results

You can choose cloud vs private when running. Call `POST /engines/resources` to check which workers match an app's GPU and RAM requirements before you submit a run — see [Engines API](/docs/api/rest/engines#match-engines-to-resource-requirements).

See [Tasks](/docs/concepts/tasks) for queue timeout and failure behavior.

### When no worker is available yet

If every matching worker is busy, offline, or does not meet the app's resource requirements (GPU type, VRAM, CPU), the task stays in **Queued** instead of failing immediately. The scheduler retries as workers come online.

| Situation | What happens |
|-----------|----------------|
| Workers are temporarily busy or restarting | Task waits in the queue; it runs when capacity returns |
| App forbids the infra you chose (e.g. cloud-only app with `infra: "private"`) | Task fails right away with a clear error |
| No matching worker exists (wrong GPU, VRAM, or infra) | Task fails after about **10 minutes** with `no active workers with matching resources` |
| Matching cloud workers exist but are reserved for other apps (`min_concurrency`) | Task fails after about **10 minutes** with `no available workers — capacity reserved for higher-priority apps` |
| Workers exist but stay busy or unavailable | Task fails after roughly **24 hours** in the queue with `task expired while waiting for available capacity, please retry` |

Some store apps have a **`min_concurrency`** floor: a cloud worker holding the app is not evicted for another app while `min_concurrency` or fewer workers hold it. The floor protects workers that already have the app loaded; it does not start or pre-load workers. This only affects **cloud** scheduling; private workers are not reserved this way. Check `min_concurrency` on the [store listing](/docs/api/rest/store#get-app-store-listing) if you see the reserved-capacity error.

On **unified-memory GPUs** (for example NVIDIA DGX Spark / GB10 or Jetson), drivers may report **0 dedicated VRAM** even though CUDA works because CPU and GPU share one memory pool. The scheduler treats available system RAM (minus an OS reserve) as schedulable GPU capacity on those hosts so VRAM requirements can still match.

For private runs, confirm your [engine is online](https://app.inference.sh/engines) and workers are configured in [Configuration](/docs/private/config). If the engine briefly shows `disconnected` (network blip or API maintenance), in-flight tasks are not failed immediately — they keep going on your hardware while the engine has reconnect grace (about five minutes). See [Engine status](/docs/api/rest/engines#get-engine) for finalization behavior. For cloud runs, queued tasks usually start within seconds unless the platform is under heavy load.

→ [Task status and queuing](/docs/concepts/tasks#task-status) · [Tasks API](/docs/api/rest/tasks)

---

## Engines

An **engine** manages workers on your hardware — either machines you install the engine on, or a **cloud engine** rented from the workspace ([provision a cloud engine](/docs/private/managing-engines#provision-a-cloud-engine)).

Install the engine on your server, and your GPUs become available for tasks.

Route tasks to your hardware with `infra: "private"` in the API or the **Private** toggle in the workspace runner. Use `infra: "private_first"` to prefer your private workers but fall back to cloud when none are available. Agents inherit the same infra setting when they call tools.

| Guide | Topic |
|-------|-------|
| [Why private?](/docs/private/why) | When to use your own workers |
| [Installing the engine](/docs/private/installing) | Engine setup on Linux and macOS |
| [Configuration](/docs/private/config) | Engine and worker settings |
| [Managing engines](/docs/private/managing-engines) | Drain, update, and restart |
| [Using private workers](/docs/private/using) | Workspace and API usage |

---

## Next

Now you know the concepts! Let's build.

→ [Creating an Agent](/docs/agents/creating)