Inference Logoinference.sh

Creating an App

Scaffold a new app with the CLI.

Create App

bash
1infsh app init my-app

Or interactive mode:

bash
1infsh app init

Answer the prompts:

PromptExample
Namemy-app
DescriptionProcesses images
Categoryimage
Python version3.11
GPU requiredyes or no

What's Created

code
1my-app/2├── inf.yml           # Configuration3├── inference.py      # Your code4├── requirements.txt  # Python dependencies5├── packages.txt      # System dependencies (optional)6├── skills/           # AI coding agent guidance7└── README.md

Files Explained

FilePurpose
inf.ymlApp settings, resources
inference.pyYour app logic
requirements.txtPython packages (pip)
packages.txtSystem packages (apt) — optional
skills/Guidance for AI coding assistants

Dependencies

Python Packages (requirements.txt)

code
1torch>=2.02transformers3accelerate

System Packages (packages.txt)

For system-level dependencies (installed via apt):

code
1ffmpeg2libgl1-mesa-glx

Base Images

Apps run in containers with these base images:

TypeImageUse Case
GPUdocker.inference.sh/gpu:latest-cudaCUDA GPU apps
CPUdocker.inference.sh/cpu:latestCPU-only apps

Note: Currently only NVIDIA CUDA GPUs are supported for GPU apps.

Next

→ App Code

previouscli setupnextapp code

we use cookies

we use cookies to ensure you get the best experience on our website. for more information on how we use cookies, please see our cookie policy.

by clicking "accept", you agree to our use of cookies.
learn more.