Inference Logoinference shell

Installing the Engine

Set up private workers on your hardware.

Requirements

Linux

  • Ubuntu 20.04+ recommended
  • Docker installed and running
  • NVIDIA GPU with drivers (for GPU tasks)
  • Outbound internet access

macOS

On macOS, the engine uses a bare runtime by default — it runs the visor worker as a native process without Docker. Apple Silicon GPU tasks use Metal (MPS) when the app supports it.

Docker is not required on macOS unless you set runtime: docker in the engine config. Linux production deployments should continue to use Docker.

Install

bash
1curl -fsSL https://engine.inference.sh/install.sh | sh

The installer places the inferencesh-engine binary in ~/.local/bin (user install) or /usr/local/bin (root install) and creates an engine alias symlink. Ensure the install directory is on your PATH.

If Docker is not installed yet on Linux:

bash
1engine setup docker

Configure

On first start, the engine prompts for an API key when api_key is missing from config. You can also pre-configure ~/.config/inferencesh/engine.yml before starting — see Configuration.

yaml
1api_url: https://api.inference.sh2api_key: inf_your_key3name: my-server

Get an API key with engines:write scope at Settings → API Keys.

Start

bash
1engine serve

engine serve and bare engine both run the worker daemon. Pass flags such as --name, --api-key, and --api-url to override config for a single run.

Verify

Check Engines in the workspace.

Your engine should appear as online.

Run as service

For production, run the daemon under a process manager. Example systemd unit (/etc/systemd/system/inferencesh-engine.service):

ini
1[Unit]2Description=inference.sh engine3After=network-online.target docker.service45[Service]6Type=simple7User=inferencesh8ExecStart=/usr/local/bin/inferencesh-engine serve9Restart=on-failure10Environment=INFSH_NO_AUTOUPDATE=11112[Install]13WantedBy=multi-user.target

Adjust User, ExecStart, and paths to match your install. Then:

bash
1sudo systemctl daemon-reload2sudo systemctl enable inferencesh-engine3sudo systemctl start inferencesh-engine

Updating the engine

The engine checks dist.inference.sh for new releases on startup and can update automatically. To upgrade safely from the workspace or your own code, see Engines API.

On the host:

bash
1engine update                  # latest release2engine update --version v1.10.15   # specific version3engine rollback               # previous version (if saved)

Disable startup auto-update with --disable-auto-update or INFSH_NO_AUTOUPDATE=1.

Next

→ Configuration

previouswhy private?nextconfiguration

we use cookies

we use cookies to ensure you get the best experience on our website. for more information on how we use cookies, please see our cookie policy.

by clicking "accept", you agree to our use of cookies.
learn more.