Inference Logoinference.sh

Configuration

Configure workers on your engine.

Config file

Located at ~/.inference/config.yml:

yaml
1api_url: https://api.inference.sh2api_key: inf_your_key3 4name: my-server5 6workers:7  cpu:8    count: 29  gpu:10    - gpus: [0]11    - gpus: [1]

CPU workers

For tasks that don't need GPU:

yaml
1workers:2  cpu:3    count: 4

GPU workers

One worker per GPU:

yaml
1workers:2  gpu:3    - gpus: [0]4    - gpus: [1]5    - gpus: [2]

Multi-GPU workers

For large models needing multiple GPUs:

yaml
1workers:2  gpu:3    - gpus: [0, 1]  # Uses both

Engine name

yaml
1name: gpu-server-1

Shows in the workspace to identify this machine.

After changes

Restart the engine:

bash
1sudo systemctl restart inference-engine

Next

→ Using Private Workers

previousinstalling the enginenextusing private workers

we use cookies

we use cookies to ensure you get the best experience on our website. for more information on how we use cookies, please see our cookie policy.

by clicking "accept", you agree to our use of cookies.
learn more.