Inference Logoinference.sh

Multi-Function Apps

Expose multiple entry points from a single app to share resources and reduce cold starts.

Why Multi-Function?

Instead of deploying separate apps for related operations, multi-function apps let you:

  • Share model loading — Load heavy models once, use across functions
  • Reduce cold starts — Warm workers serve any function immediately
  • Logical grouping — Keep related operations together

Example use cases:

  • Image models: generate, upscale, inpaint
  • LLMs: chat (history-aware) vs completion (raw text)
  • Data pipelines: process, validate, export

Function Discovery

Functions are automatically discovered from public methods on your App class.

Discovery Rules

A method becomes a function if it meets all these criteria:

RuleDescription
PublicName doesn't start with _
Not reservedNot setup or unload (lifecycle methods)
Has type hintsBoth input parameter and return type annotated
Pydantic inputInput type extends BaseAppInput
Pydantic outputReturn type extends BaseAppOutput
Single parameterExactly one parameter (besides self)

Example

python
1from inferencesh import BaseApp, BaseAppInput, BaseAppOutput23# Input schemas (one per function)4class GenerateInput(BaseAppInput):5    prompt: str6    steps: int = 3078class UpscaleInput(BaseAppInput):9    image_uri: str10    scale: float = 2.01112# Output schemas (can share if identical)13class ImageOutput(BaseAppOutput):14    uri: str15    width: int16    height: int1718class App(BaseApp):19    async def setup(self, config):20        # Load model once, used by all functions21        self.model = load_diffusion_model()2223    # ✓ Discoverable: public, typed, Pydantic models24    async def generate(self, input: GenerateInput) -> ImageOutput:25        image = self.model.generate(input.prompt, steps=input.steps)26        return ImageOutput(uri=save(image), width=512, height=512)2728    # ✓ Discoverable29    async def upscale(self, input: UpscaleInput) -> ImageOutput:30        image = load(input.image_uri)31        result = self.model.upscale(image, input.scale)32        return ImageOutput(uri=save(result), width=1024, height=1024)3334    # ✗ Not discoverable: starts with underscore35    async def _helper(self, data):36        pass3738    # ✗ Not discoverable: no type hints39    async def process(self, input):40        pass

Input and Output Schemas

Each function has independent typed input/output schemas:

python
1class ChatInput(BaseAppInput):2    messages: list[dict]3    max_tokens: int = 100045class ChatOutput(BaseAppOutput):6    response: str7    tokens_used: int89class CompletionInput(BaseAppInput):10    prompt: str11    max_tokens: int = 10001213class CompletionOutput(BaseAppOutput):14    text: str15    tokens_used: int1617class App(BaseApp):18    async def chat(self, input: ChatInput) -> ChatOutput:19        ...2021    async def completion(self, input: CompletionInput) -> CompletionOutput:22        ...

Schema Generation

When you deploy, the system generates functions_schema.json:

json
1{2  "generate": {3    "input_schema": { ... },4    "output_schema": { ... }5  },6  "upscale": {7    "input_schema": { ... },8    "output_schema": { ... }9  }10}

Pydantic Compatibility

Both Pydantic V1 and V2 are supported:

python
1# Pydantic V12from pydantic import BaseModel34# Pydantic V25from pydantic import BaseModel67# Both work — the system detects the version automatically

Calling Functions

Specify the function parameter when calling the app.

CLI

bash
1# Call specific function2infsh app run user/image-model --function upscale --input '{"image_uri": "inf://...", "scale": 4.0}'34# Call default function (run)5infsh app run user/image-model --input '{"prompt": "a sunset"}'

Python SDK

python
1from inferencesh import Inference23client = Inference(api_key="...")45# Call specific function6result = client.run({7    "app": "user/image-model@abc123",8    "function": "upscale",9    "input": {"image_uri": "inf://...", "scale": 4.0}10})1112# Call default function13result = client.run({14    "app": "user/image-model@abc123",15    "input": {"prompt": "a sunset"}16})

JavaScript SDK

typescript
1import { Inference } from '@anthropic/inference-sdk';23const client = new Inference({ apiKey: '...' });45// Call specific function6const result = await client.run({7  app: 'user/image-model@abc123',8  function: 'upscale',9  input: { image_uri: 'inf://...', scale: 4.0 }10});

REST API

bash
1curl -X POST https://api.inference.sh/apps/run \2  -H "Authorization: Bearer $API_KEY" \3  -H "Content-Type: application/json" \4  -d '{5    "app": "user/image-model@abc123",6    "function": "upscale",7    "input": {"image_uri": "inf://...", "scale": 4.0}8  }'

Default Function

If no function is specified, the system calls:

  1. A method named run (if it exists and is discoverable)
  2. Otherwise, returns an error
python
1class App(BaseApp):2    # This becomes the default function3    async def run(self, input: GenerateInput) -> ImageOutput:4        return await self.generate(input)56    async def generate(self, input: GenerateInput) -> ImageOutput:7        ...

Common Patterns

Multiple Related Operations

python
1class App(BaseApp):2    async def setup(self, config):3        self.model = load_model()  # Shared across all functions45    async def generate(self, input: GenerateInput) -> ImageOutput:6        """Generate from text prompt."""7        ...89    async def img2img(self, input: Img2ImgInput) -> ImageOutput:10        """Transform existing image."""11        ...1213    async def inpaint(self, input: InpaintInput) -> ImageOutput:14        """Fill masked regions."""15        ...1617    async def upscale(self, input: UpscaleInput) -> ImageOutput:18        """Increase resolution."""19        ...

Chat vs Completion Modes

python
1class App(BaseApp):2    async def chat(self, input: ChatInput) -> ChatOutput:3        """Conversation with message history."""4        messages = input.messages5        response = self.model.chat(messages)6        return ChatOutput(response=response)78    async def completion(self, input: CompletionInput) -> CompletionOutput:9        """Single-shot text completion."""10        text = self.model.complete(input.prompt)11        return CompletionOutput(text=text)

Process / Validate / Export Pipeline

python
1class App(BaseApp):2    async def process(self, input: ProcessInput) -> ProcessOutput:3        """Transform raw data."""4        ...56    async def validate(self, input: ValidateInput) -> ValidateOutput:7        """Check data quality."""8        ...910    async def export(self, input: ExportInput) -> ExportOutput:11        """Format for delivery."""12        ...

Validation and Debugging

Check Discovery Locally

Run validation to see discovered functions:

bash
1infsh app validate

Output:

code
1✓ discovered 3 functions: [generate, upscale, inpaint]

Why Methods Get Skipped

Run with verbose mode to see skip reasons:

bash
1infsh app validate --verbose

Output:

code
1Checking method: generate2  ✓ public method3  ✓ not reserved4  ✓ has return type hint5  ✓ has 1 parameter6  ✓ parameter has type hint7  ✓ input type is Pydantic model8  ✓ output type is Pydantic model9  → DISCOVERED1011Checking method: _helper12  ✗ private method (starts with _)13  → SKIPPED1415Checking method: process16  ✗ missing return type hint17  → SKIPPED

Test Functions Locally

bash
1# Test specific function2infsh app dev --function upscale --input '{"image_uri": "...", "scale": 2.0}'34# Test default function5infsh app dev --input '{"prompt": "test"}'

Next

previousdeployingnextsessions

we use cookies

we use cookies to ensure you get the best experience on our website. for more information on how we use cookies, please see our cookie policy.

by clicking "accept", you agree to our use of cookies.
learn more.