Set up the inference shell proxy in SvelteKit applications.
Installation
bash
1npm install @inferencesh/sdkBasic Setup
Create src/routes/api/inference/proxy/+server.ts:
typescript
1import { createHandler } from "@inferencesh/sdk/proxy/svelte";23const handler = createHandler({ allowedEndpoints: ["my-team/*"] });45export const GET = handler;6export const POST = handler;The proxy forwards app runs and agent chats only; allowedEndpoints limits them to your apps and agents. See options.
[!WARNING] Anyone who can reach the proxy route runs apps and agents on your API key. In production set
allowedEndpointsandisAuthenticated(see what the proxy forwards), and create a Run only key in a workspace that holds nothing else.
Environment Variable
Add to your .env:
bash
1INFERENCE_API_KEY="inf_your_key_here"Make sure to expose it in svelte.config.js if needed:
javascript
1import { defineConfig } from "vite";2import { sveltekit } from "@sveltejs/kit/vite";34export default defineConfig({5 plugins: [sveltekit()],6 define: {7 // Expose to server-side code8 "process.env.INFERENCE_API_KEY": JSON.stringify(process.env.INFERENCE_API_KEY),9 },10});Client Configuration
typescript
1// src/lib/inference.ts2import { inference } from "@inferencesh/sdk";34export const client = inference({5 proxyUrl: "/api/inference/proxy"6});Use in components:
svelte
1<script lang="ts">2 import { client } from "$lib/inference";34 let prompt = "";5 let result = null;6 let loading = false;78 async function generate() {9 loading = true;10 result = await client.run({11 app: "infsh/flux",12 input: { prompt }13 });14 loading = false;15 }16</script>1718<input bind:value={prompt} />19<button on:click={generate} disabled={loading}>20 {loading ? "Generating..." : "Generate"}21</button>2223{#if result}24 <img src={result.output.url} alt={prompt} />25{/if}With Authentication
isAuthenticated gets the RequestEvent; false answers 401:
typescript
1// src/routes/api/inference/proxy/+server.ts2import { createHandler } from "@inferencesh/sdk/proxy/svelte";34const handler = createHandler({5 allowedEndpoints: ["my-team/*"],6 isAuthenticated: (event) => Boolean(event.locals.user), // set by hooks.server.ts7});89export const GET = handler;10export const POST = handler;Custom API Key Resolution
typescript
1import { createHandler } from "@inferencesh/sdk/proxy/svelte";23const handler = createHandler({4 resolveApiKey: async () => {5 // Load from secrets manager or database6 return await getSecret("INFERENCE_API_KEY");7 }8});910export const GET = handler;11export const POST = handler;Streaming
The SvelteKit handler supports streaming responses for real-time task updates.
typescript
1import { client } from "$lib/inference";23const result = await client.run(4 { app: "infsh/flux", input: { prompt } },5 {6 onUpdate: (update) => {7 console.log("Status:", update.status);8 }9 }10);