Graceful Cancellation

Apps typically process requests until completion. However, users may cancel long-running tasks. Implementing graceful cancellation lets your app stop at a clean point, release resources, and keep its loaded state for the next task.

How a cancel reaches your app

When a task is cancelled, the platform tells the running app two ways, at the same time:

1. self.context.cancel_requested (Python) / this.context.cancelRequested (Node.js) becomes true. In Python this flag is set from a separate thread, so it is visible even while your code is blocking the event loop in a synchronous model call.
2. Your on_cancel() / onCancel() method is called, if you define one. It runs on the event loop, so it waits until your run yields or awaits.

Both are delivered while the run is in progress, so your app can react in the middle of its work.

The Cancel Hook

Define a cancel method in your App class, or poll the flag, or both.

1class App(BaseApp):2    async def setup(self, config):3        self.cancel_flag = False45    async def on_cancel(self):6        """Called when the user cancels the task"""7        print("Cancellation requested...")8        self.cancel_flag = True9        return True1011    async def run(self, input):12        self.cancel_flag = False13        for i in range(100):14            if self.cancel_flag or self.context.cancel_requested:15                print("Stopping work...")16                break17            await self.heavy_computation(i)

Checking self.context.cancel_requested is enough on its own: the platform sets it whether or not you define on_cancel().

Returning what you have

An app that stops can still return a result: the text generated so far, the frames rendered, the conversation of a live session. If it returns within the grace period, the task completes with that result, and it is billed from its output_meta like any other result. Cancelling asked the app to stop; what it returned is what the task produced.

Define on_cancel() / onCancel() to do this: it tells the platform the app stops itself. Without it the platform stops the run for you (it stops reading a generator's yields) and the task ends cancelled.

1async def on_cancel(self):2    return True                                    # this app stops itself: see run34async def run(self, input_data: AppInput) -> AppOutput:5    text, tokens = "", 06    async for piece in self.model.generate(input_data.prompt):7        if self.context.cancel_requested:8            break                                  # stop, and keep what there is9        text += piece.text10        tokens += piece.tokens11    return AppOutput(text=text, output_meta=OutputMeta(outputs=[TextMeta(tokens=tokens)]))

An app that raises instead, or returns nothing in time, ends cancelled with no output and nothing billed. A live function needs no on_cancel: a cancel ends its stream the way a disconnect does, and the result it returns completes the task.

What happens if the app does not stop

A cancel gives your app part of the cancel timeout (default 10 seconds) to stop on its own. If the run is still going after that, the platform ends it by restarting the app process:

• The run, and any GPU memory it holds, ends with the process.
• The container, its environment and cached model weights survive. The next task on this worker runs your setup() again and reloads the model from the local cache, instead of paying for a full cold start.

The same happens when a task is cancelled during setup(), which cannot be interrupted, and when a task is force-cancelled.

Stopping on your own is cheaper for the next task: a cancelled app that honours the cancel keeps its container, its process and its loaded model.

A task that is cancelled never leaks into the next one. Work still executing after a cancel is ended before the worker takes another task, and nothing it prints or produces after the cancel is reported.

Best Practices

1. Check Frequently: In loops, check self.context.cancel_requested (or your own flag) at the start of every iteration, and between the steps of a long synchronous call where you can.

2. Clean Up: Close database connections, delete temporary files, or free GPU memory before returning.

3. Return Quickly: The cancel handler should be fast. Do not block execution there; just set a flag or signal an event.

4. Force Kill: Callers can skip the grace period and end the run immediately:

   • REST API: POST /tasks/:id/cancel with {"force": true} — see Cancel task
   • CLI: belt task cancel <task-id> --force

   Use force kill only when graceful cancellation is stuck or too slow: it always costs the next task a setup().