Inference Logoinference.sh

Installing the Engine

Set up private workers on your hardware.

Requirements

Linux

  • Ubuntu 20.04+ recommended
  • Docker installed and running
  • NVIDIA GPU with drivers (for GPU tasks)
  • Outbound internet access

macOS

On macOS, the engine uses a bare runtime by default — it runs the visor worker as a native process without Docker. Apple Silicon GPU tasks use Metal (MPS) when the app supports it.

Docker is not required on macOS unless you set runtime: docker in the engine config. Linux production deployments should continue to use Docker.

Install

bash
1curl -fsSL https://engine.inference.sh/install.sh | sh

Initialize

bash
1inference-engine init

Enter your API key when prompted.

Start

bash
1inference-engine start

Verify

Check Engines in the workspace.

Your engine should appear as online.

Run as service

For production, run as a systemd service:

bash
1sudo systemctl enable inference-engine2sudo systemctl start inference-engine

Updating the engine

The engine checks dist.inference.sh for new releases on startup and can update automatically. To upgrade safely from the workspace or your own code, see Engines API.

On the host:

bash
1inference-engine update                  # latest release2inference-engine update --version v1.10.15   # specific version3inference-engine rollback               # previous version (if saved)

Disable startup auto-update with --disable-auto-update or INFSH_NO_AUTOUPDATE=1.

Next

→ Configuration

previouswhy private?nextconfiguration

we use cookies

we use cookies to ensure you get the best experience on our website. for more information on how we use cookies, please see our cookie policy.

by clicking "accept", you agree to our use of cookies.
learn more.