# Graceful Cancellation
> Apps typically process requests until completion. However, users may cancel long-running tasks. Implementing graceful cancellation lets your app stop at a clean point, release resources, and keep its ...
**URL:** https://inference.sh/docs/extend/cancellation
**Last updated:** 2026-09-24
---
Apps typically process requests until completion. However, users may cancel long-running tasks. Implementing graceful cancellation lets your app stop at a clean point, release resources, and keep its loaded state for the next task.
## How a cancel reaches your app
When a task is cancelled, the platform tells the running app two ways, at the same time:
1. `self.context.cancel_requested` (Python) / `this.context.cancelRequested` (Node.js) becomes `true`. In Python this flag is set from a separate thread, so it is visible even while your code is blocking the event loop in a synchronous model call.
2. Your `on_cancel()` / `onCancel()` method is called, if you define one. It runs on the event loop, so it waits until your `run` yields or awaits.
Both are delivered while the run is in progress, so your app can react in the middle of its work.
## The Cancel Hook
Define a cancel method in your `App` class, or poll the flag, or both.
```python Python
class App(BaseApp):
async def setup(self, config):
self.cancel_flag = False
async def on_cancel(self):
"""Called when the user cancels the task"""
print("Cancellation requested...")
self.cancel_flag = True
return True
async def run(self, input):
self.cancel_flag = False
for i in range(100):
if self.cancel_flag or self.context.cancel_requested:
print("Stopping work...")
break
await self.heavy_computation(i)
```
```javascript Node.js
export class App {
constructor() {
this.cancelFlag = false;
}
async onCancel() {
console.log("Cancellation requested...");
this.cancelFlag = true;
return true;
}
async *run(inputData) {
this.cancelFlag = false;
for (let i = 0; i < 100; i++) {
if (this.cancelFlag || this.context.cancelRequested) {
console.log("Stopping work...");
break;
}
yield await this.heavyComputation(i);
}
}
}
```
Checking `self.context.cancel_requested` is enough on its own: the platform sets it whether or not you define `on_cancel()`.
## Returning what you have
An app that stops can still return a result: the text generated so far, the frames rendered, the conversation of a live session. If it returns within the grace period, the task **completes** with that result, and it is billed from its `output_meta` like any other result. Cancelling asked the app to stop; what it returned is what the task produced.
Define `on_cancel()` / `onCancel()` to do this: it tells the platform the app stops itself. Without it the platform stops the run for you (it stops reading a generator's yields) and the task ends cancelled.
```python Python
async def on_cancel(self):
return True # this app stops itself: see run
async def run(self, input_data: AppInput) -> AppOutput:
text, tokens = "", 0
async for piece in self.model.generate(input_data.prompt):
if self.context.cancel_requested:
break # stop, and keep what there is
text += piece.text
tokens += piece.tokens
return AppOutput(text=text, output_meta=OutputMeta(outputs=[TextMeta(tokens=tokens)]))
```
```javascript Node.js
async onCancel() {
return true; // this app stops itself: see run
}
async run(input) {
let text = "", tokens = 0;
for await (const piece of this.model.generate(input.prompt)) {
if (this.context.cancelRequested) break; // stop, and keep what there is
text += piece.text;
tokens += piece.tokens;
}
return { text, output_meta: { outputs: [{ type: "text", tokens }] } };
}
```
An app that raises instead, or returns nothing in time, ends **cancelled** with no output and nothing billed. A [live function](/docs/extend/streaming) needs no `on_cancel`: a cancel ends its stream the way a disconnect does, and the result it returns completes the task.
## What happens if the app does not stop
A cancel gives your app part of the cancel timeout (default 10 seconds) to stop on its own. If the run is still going after that, the platform ends it by **restarting the app process**:
- The run, and any GPU memory it holds, ends with the process.
- The container, its environment and cached model weights survive. The next task on this worker runs your `setup()` again and reloads the model from the local cache, instead of paying for a full cold start.
The same happens when a task is cancelled during `setup()`, which cannot be interrupted, and when a task is force-cancelled.
Stopping on your own is cheaper for the next task: a cancelled app that honours the cancel keeps its container, its process and its loaded model.
A task that is cancelled never leaks into the next one. Work still executing after a cancel is ended before the worker takes another task, and nothing it prints or produces after the cancel is reported.
## Best Practices
1. **Check Frequently**: In loops, check `self.context.cancel_requested` (or your own flag) at the start of every iteration, and between the steps of a long synchronous call where you can.
2. **Clean Up**: Close database connections, delete temporary files, or free GPU memory before returning.
3. **Return Quickly**: The cancel handler should be fast. Do not block execution there; just set a flag or signal an event.
4. **Force Kill**: Callers can skip the grace period and end the run immediately:
- **REST API:** `POST /tasks/:id/cancel` with `{"force": true}` — see [Cancel task](/docs/api/rest/tasks#cancel-task)
- **CLI:** `belt task cancel --force`
Use force kill only when graceful cancellation is stuck or too slow: it always costs the next task a `setup()`.