Troubleshooting

Common issues and solutions for inference shell apps.

Task stuck in Queued

Symptom: A run stays in Queued and never moves to Running.

Check:

1. Cloud run — Usually resolves when a cloud worker is free. Heavy load or maintenance can add a short wait.
2. Private run — Open Engines. The engine must be online and workers must match the app's GPU/CPU requirements. See Using private workers and Configuration.
3. Resource mismatch — The app may require more VRAM or a specific GPU type than any connected worker provides.
4. Infra mismatch — Some apps only allow cloud or only private; wrong infra fails immediately instead of queuing.

Tasks wait in the queue for up to roughly 24 hours before failing with task expired while waiting for available capacity, please retry. If no worker matches the app's resource requirements at all, failure happens sooner (~10 minutes) with no active workers with matching resources. See Task status and Workers.

CLI: Use belt task get <id> -f to watch status until the task starts or fails. See CLI setup.

Platform task failures

Symptom: Task status is Failed with a short, generic error like task interrupted, please retry or task could not be dispatched, please retry.

Cause: The platform hit a transient infrastructure issue — worker disconnect, engine reconnect, dispatch retry exhaustion, or similar. These messages are intentionally user-friendly; internal diagnostics are not exposed in the error field.

What to do:

1. Retry the run: most platform failures are transient.
2. Check task logs: app-level errors (validation, OOM, model failures) appear with your app's own message and are not sanitized.
3. Session runs: if you see session worker busy too long, wait for the prior call on that session to finish or serialize session calls. See Sessions: task failures.
4. Secrets: failed to prepare secrets for this app, please check your secret configuration means a required workspace secret is missing or invalid. Add or fix secrets in vaults → secrets.

For the full list of platform error strings, see Task failure messages.

Access denied (IP blocked)

Symptom: Every API request returns HTTP 403 with:

1{2  "message": "Access denied. This attempt has been logged. If you believe this is a mistake, please disconnect any VPN or proxy and try again, or contact [email protected]."3}

Cause: Your IP address was blocked for abusive or malicious activity. The check runs before authentication on almost every route. Some health and status endpoints stay reachable.

What to do:

1. Disconnect VPN or proxy services and retry from a direct connection.
2. If the block persists, contact [email protected] with your public IP address and what you were trying to do.

This is separate from permission 403 errors (forbidden, feature_not_available) and from magic-link rate limits (429). See REST overview: Pre-authentication access controls.

Service not available in your region

Symptom: Every API request returns HTTP 451 with plain text:

1service not available in your region

Cause: inference shell blocks API access from countries and regions under OFAC sanctions. The check runs before authentication on almost every route. Blocked countries are Cuba (CU), Iran (IR), North Korea (KP), Russia (RU), and Syria (SY). Within Ukraine, Crimea, Sevastopol, Donetsk, and Luhansk are also blocked.

What to do:

1. Disconnect VPN or proxy services that exit in a blocked country or region and retry from a permitted location.
2. If you believe the geo lookup is wrong, contact [email protected] with your public IP address and location.

This is separate from IP abuse blocks (403) and from plan-limit errors. See REST overview: Pre-authentication access controls.

Signup blocked (disposable email)

Symptom: Sign-up fails with Temporary email addresses are not allowed. Please use a permanent email address. API responses use code disposable_email (HTTP 400) on magic link send and OAuth callbacks.

Cause: inference shell blocks new accounts on known disposable or temporary email domains. The check runs when a new user record would be created, not when an existing account signs in.

What to do:

1. Sign up with a permanent email address.
2. If your account was registered on such a domain before the restriction, sign in with your original method. Existing accounts are not blocked.
3. On a first SAML login, the same rejection surfaces as a generic user_creation_failed error instead of disposable_email. Retry with a permanent address.

Plus (+) aliases: Magic link sign-up currently allows + tags (for example [email protected]). If you later see invalid_email on magic link send, the platform may have turned on plus-alias blocking for new accounts on your email provider—use your base address, or sign in with Gmail/Googlemail where + tags fold into one mailbox. Existing accounts are not affected.

→ REST overview: Signup email restrictions · Magic link errors

CLI billing or plan limit errors

Symptom: belt or infsh prints a message about insufficient balance, upgrading your plan, or a link to billing settings.

Cause: The API returned 402 Payment Required (credits) or 403 Forbidden with an entitlement error (upgrade_available in the error metadata).

What to do:

1. Insufficient balance: add credits at settings → workspace → billing. The CLI includes the billing URL in the error message.
2. Plan limit: upgrade at settings → workspace → subscription when the error links to the subscription page.
3. Addon entitlement: subscribe at settings → workspace → add-ons when the error links to the add-ons page (for example a blocked feature:seedance store app).
4. Not logged in: run belt login (or infsh login) if you see "not logged in or session expired".

These messages come from the API error body; other failures use RFC 9457 problem+json.

Deploy: integrations: in inf.yml

Symptom: Loading or deploying fails with inf.yml: the integrations: key was renamed credentials: — rename it and deploy again.

Cause: CLI v1.18.59 and later no longer read the old integrations: block. Requirements must live under credentials:.

What to do:

1. Rename the top-level key to credentials: (same list shape).
2. Run belt app credentials list to confirm capability key values.
3. Redeploy.

See Credentials — Declaring credentials.

Deploy: admin_session_required or another namespace

Symptom: belt app deploy fails with HTTP 403 and a message that starts Deploying into namespace "…" needs admin access.

Cause: inf.yml names a namespace other than your active workspace, and your account has the platform staff role. Deploying into another namespace needs admin access; the deploy is refused rather than moved into your workspace. The error code says what is missing:

To deploy into your own workspace, set namespace to your workspace or remove it. A member without the staff role who names another namespace deploys into their own workspace.

→ Deploying: Namespace

Deploy: unknown credential keys

Symptom: belt app deploy fails during credential validation with unknown credential keys: [...] and suggests belt app credentials list.

Cause: A credentials: entry includes a key that is not a registered platform capability. Capability keys must match the list from belt app credentials list (or GET /integrations/capabilities).

What to do:

1. Fix typos in key (for example google.sheet vs google.sheets).

2. For BYOK API keys tied to a provider credential, use a provider-only entry instead of inventing a capability name:

   yamlcopy

   1credentials:2  - provider: acme3    secrets: [ACME_API_KEY]

   Deploy does not treat provider-only rows as capabilities; the API checks provider requirements when the version is created. See Credentials — Provider-only entries.

3. If you set both provider and key, only key is validated as a capability—the value must still appear in belt app credentials list.

Tasks rejected during engine maintenance

Symptom: New private tasks fail immediately or stay queued while you update or drain an engine.

Cause: While an engine is draining (or receiving an update), it rejects new work with reason engine_draining until in-flight tasks finish and the engine restarts.

What to do:

1. Wait for running tasks on that engine to complete, or cancel them if you need a faster turnaround.
2. Use Drain before maintenance and Update when you want a safe binary upgrade — see Engines API.
3. Route urgent work to cloud infra or another online engine with infra: "cloud" or a different private engine.

Plan limits and billing errors

Symptom: API or CLI calls fail with HTTP 402 or 403, or messages about limits, balance, or upgrading your plan.

Plan usage limit (limit_exceeded, HTTP 402)

You hit a subscription limit — for example too many API keys, connectors, knowledge bases, private apps, or storage.

What to do:

1. The web app opens an upgrade modal with a recommended plan, the cheapest tier that lifts the blocked limit. It shows current usage vs your cap when the cap is greater than zero.
2. Open settings → workspace → subscription to compare all plans and change your subscription.
3. Delete or reduce usage (unused API keys, old knowledge entries, large files).
4. Inspect limits programmatically: GET /entitlements for limits and GET /entitlements/usage for current counts — see Entitlements API.

Feature not on your tier (feature_not_available, HTTP 403)

The operation requires a capability your plan does not include (for example feature:publish_apps or feature:webhooks).

The web app's upgrade modal lists higher tiers and recommends the cheapest plan that enables the feature. Upgrade at settings → workspace → subscription or contact support if you believe the feature should already be enabled.

Insufficient balance (payment_required, HTTP 402)

Task and agent runs charge prepaid credits. When balance is too low, POST /run and agent message endpoints return payment_required. They also return it when the payer's billing status (GET /billing) is not active: spend_locked, manual_hold or suspended. Those statuses refuse free runs too.

Add credits at settings → workspace → billing. A credit that takes the balance above the spend floor clears spend_locked; manual_hold and suspended need support. See Billing: Get billing account.

Payment method required (payment_method_required, HTTP 402)

Symptom: A run of an app that costs nothing for your input returns payment_method_required while the balance is too low for a paid run.

Cause: A free run needs no credits, but the workspace that pays must have a saved payment method and billing status active. The response's meta.billing_page is /settings/team/billing/payments.

What to do: Save a card under settings → workspace → billing → payments and retry. Nothing is charged for a free run. See Billing: free runs.

Entitlement lookup failed (entitlement_unavailable, HTTP 500)

Symptom: A request that checks plan limits (for example POST /files, GET /entitlements, or GET /entitlements/usage) fails with could not verify plan limits, try again and code entitlement_unavailable.

Cause: The API could not read your workspace's entitlement rows. This is a transient server error, not a plan cap or a disabled feature. The API fails closed: it denies the operation until entitlements can be verified again.

What to do:

1. Retry the request after a short delay.
2. If it persists, contact support. Upgrading your plan or adding credits does not fix it.

CLI messages

The belt / infsh CLI prints the API detail message for most errors. When the response includes upgrade_available: true in the entitlement metadata under top-level meta, the CLI appends an upgrade URL. When addon_plan_id is also present, the URL points to the add-ons page; otherwise it points to the subscription page. For HTTP 402 without that metadata, the CLI suggests the billing URL for credits instead. Check whether the failure is a plan limit (entitlements), addon (feature gate), or balance (credits).

→ REST overview — Billing and plan limits · Entitlements API

Import Errors

"ModuleNotFoundError" in Production

Common cause: a package your code imports is not in requirements.txt. The Python base environment holds only build tools, pydantic and inferencesh; torch, accelerate, huggingface_hub, transformers and diffusers are not preinstalled. A version that ran before may fail once it is set up again on a new worker, or when you deploy a new version. See Python environment.

Solutions:

1. Add the missing package to requirements.txt and deploy again. transformers and diffusers need accelerate for device_map, low_cpu_mem_usage and enable_model_cpu_offload().

2. Add __init__.py files to all packages

3. Add current directory to Python path:

1import sys, os2sys.path.append(os.path.dirname(os.path.abspath(__file__)))

1. For local packages, use editable installs:

1-e ./local_package_directory

Memory Issues

"CUDA out of memory"

Solutions:

1. Reduce batch size
2. Use mixed precision: model.to(dtype=torch.float16)
3. Enable gradient checkpointing: model.gradient_checkpointing_enable()
4. Clear cache: torch.cuda.empty_cache()
5. Increase VRAM in inf.yml

Memory Leaks

Clean up after each request:

1import gc, torch23async def run(self, input_data):4    result = self.process(input_data)5    if torch.cuda.is_available():6        torch.cuda.empty_cache()7    gc.collect()8    return result

Device Errors

"Expected all tensors to be on the same device"

Ensure all tensors are on the same device:

1input_tensor = input_tensor.to(self.device)

"CUDA not available"

1. Check inf.yml GPU requirements:

1resources:2  gpu:3    count: 14    vram: 24  # 24GB

1. Use device detection (add accelerate to requirements.txt):

1from accelerate import Accelerator2device = Accelerator().device

Model Loading Errors

"File not found" After Download

Don't assume file paths:

1model_path = snapshot_download(repo_id="org/model")2config_path = os.path.join(model_path, "config.yaml")3if os.path.exists(config_path):4    # Load config

"Token required for gated model"

Add HF_TOKEN to secrets:

1secrets:2  - key: HF_TOKEN3    description: HuggingFace token for gated models

File Path Issues

Temporary Files Deleted Too Early

Use delete=False:

1with tempfile.NamedTemporaryFile(suffix='.jpg', delete=False) as tmp:2    output_path = tmp.name

Path Separators

Use os.path.join:

1# ✅ Good2path = os.path.join("models", "config", "settings.json")

Dependency Issues

Version Conflicts

Pin compatible versions:

1torch==2.6.02numpy>=1.23.5,<2

Flash Attention Build Errors

Use pre-built wheels in requirements2.txt

Debug Mode

Add logging:

1import logging2logging.basicConfig(level=logging.DEBUG)34async def setup(self, config):5    logging.debug(f"Config: {config}")6    logging.info("Starting model load...")

Node.js-Specific Issues

"ERR_MODULE_NOT_FOUND"

Importing @inferencesh/sdk in plain Node: up to v0.22.1 the package's dist/ used extensionless relative imports, so import '@inferencesh/sdk' and most subpaths failed with ERR_MODULE_NOT_FOUND without a bundler. Upgrade to @inferencesh/sdk ≥ v0.23.0.

For your own ESM project files, ensure "type": "module" is in your package.json and use file extensions in relative imports:

1import { helper } from "./helper.js"; // .js required for ESM

"Cannot use import statement outside a module"

Your package.json must have:

1{2  "type": "module"3}

Native Module Build Errors

Some packages (e.g., sharp, canvas) need system libraries. Add them to packages.txt:

1libvips-dev

Claude Code belt plugin issues

Symptom: Hooks stop running, belt suggest never appears in context, or Claude Code reports that the plugin directory does not exist.

Check:

1belt plugin doctor

The doctor verifies belt CLI availability, authentication (not guest), the belt entry in Claude's installed_plugins.json, that the install path exists, that hooks/hooks.json is present, that hook scripts in bin/ use the thin shim pattern (exec belt plugin hook <event>) across Claude and Codex plugin caches, and recent hook activity in ~/.belt/hooks.log.

Fix:

1belt plugin doctor --fix

--fix reinstalls the belt plugin when the cache path is stale or missing (claude plugin uninstall belt@belt-sh-skills, then claude plugin install belt, with marketplace add as fallback), removes other stale cache directories under the plugin install parent, and patches hook scripts that still contain embedded logic instead of the thin shim format (exec belt plugin hook <event>). The same hook patching runs automatically after belt plugin install claude because marketplace caching can serve stale hook scripts. For auth issues it prints belt login guidance (guest sessions disable knowledge persistence).

When all checks pass, doctor prints All checks passed. When --fix applies repairs, it lists each change (for example fixed: removed stale cache dir abc123, fixed: reinstalled plugin (cache path was stale), or fixed: patched hook-stop.sh in <hash>), then prompts you to restart. Inside a Claude Code session, doctor suggests Run /reload-plugins or restart this session to apply changes; from a regular terminal, it suggests Restart your agent session to apply changes.

→ Claude Code plugin · CLI setup — Agent integration

Kiro belt hooks not running

Symptom: belt suggest never appears in Kiro sessions after belt plugin install kiro.

Check:

1belt plugin doctor

Look for the kiro-agent check. Belt's hooks live in ~/.kiro/agents/belt.json and run only while belt is Kiro's default agent (chat.defaultAgent in .kiro/settings/cli.json; the workspace setting wins over ~/.kiro/settings/cli.json).

Fix:

1belt plugin doctor --fix           # sets belt as default when no other agent is selected2kiro-cli agent set-default belt    # or set it yourself

If another Kiro agent is your default, copy the hooks block from ~/.kiro/agents/belt.json into that agent's config instead.

→ Coding agents: Other agent runtimes

OpenCode or Kilo: no belt suggestions on prompt

Symptom: belt suggest never appears in OpenCode or Kilo sessions after belt plugin install, or the same suggestions repeat several times per message.

Cause: OpenCode and Kilo inject belt output through a system-prompt transform rather than a direct prompt hook. Belt v1.18.43 and later pass the user prompt into the hook correctly; older plugin installs ran the hook without stdin, so belt received no prompt and printed nothing.

Fix:

1belt update                        # belt v1.18.43+ required2belt plugin install opencode       # or: belt plugin install kilo

Restart OpenCode or Kilo after reinstalling. Verify with belt plugin status — the agent should show belt hooks at user or project scope.

→ Coding agents: Other agent runtimes · CLI setup — Agent integration

Goose: belt skills missing from goose skills list

Symptom: goose skills list omits belt skills, but belt hooks still run.

Cause: A plugin.json at ~/.agents/plugins/belt/plugin.json (hand-written or from an older workaround). Goose hides plugin components when manifest paths are wrong — no error, zero exit.

Fix:

1rm ~/.agents/plugins/belt/plugin.json   # only if you added one manually2belt plugin install goose               # reinstall skills and hooks3goose skills list

If you installed before September 2026, hooks may still be under ~/.agents/plugins/belt-test/ — belt plugin install goose moves them to ~/.agents/plugins/belt/.

→ Coding agents: Goose

Stale CLI on PATH

Symptom: belt or infsh fails with a Python import error, runs an old version, or behaves differently depending on which command name you use.

Cause: An older install shadows the current binary. The usual culprit is the legacy PyPI inferencesh package, which shipped an infsh launcher.

Fix:

1belt auth doctor            # look for a warning on the path check2pip uninstall inferencesh   # if you installed the old PyPI package3which belt infsh inferencesh

Remove the stale binary, or put the directory named in the path check's hint earlier in PATH. See CLI setup: Authentication (belt auth doctor).

CLI billing and authentication errors

Symptom: belt or infsh fails with a message about balance, plan limits, or login.

See CLI setup — Billing and plan limits for details.

Next

→ Best Practices - Optimization patterns