Inference Logoinference.sh

Express

Set up the inference.sh proxy in Express.js applications.


Installation

bash
1npm install @inferencesh/sdk express

Basic Setup

typescript
1import express from "express";2import * as inferenceProxy from "@inferencesh/sdk/proxy/express";3 4const app = express();5 6// Parse JSON bodies7app.use(express.json());8 9// Mount the proxy10app.all(inferenceProxy.route, inferenceProxy.handler);11 12app.listen(3000, () => {13  console.log("Server running on port 3000");14});

The default route is /api/inference/proxy.


Environment Variable

Set your API key:

bash
1export INFERENCE_API_KEY="inf_your_key_here"

Or use a .env file with dotenv:

bash
1# .env2INFERENCE_API_KEY=inf_your_key_here
typescript
1import "dotenv/config";2import express from "express";3import * as inferenceProxy from "@inferencesh/sdk/proxy/express";4 5const app = express();6app.use(express.json());7app.all(inferenceProxy.route, inferenceProxy.handler);

Client Configuration

typescript
1import { inference } from "@inferencesh/sdk";2 3const client = inference({4  proxyUrl: "http://localhost:3000/api/inference/proxy"5});6 7const result = await client.run({8  app: "infsh/flux",9  input: { prompt: "A sunset" }10});

Custom Route

Mount at a different path:

typescript
1app.all("/my-custom-proxy/*", inferenceProxy.handler);

Update client:

typescript
1const client = inference({2  proxyUrl: "/my-custom-proxy"3});

With Middleware

Add authentication or rate limiting:

typescript
1import express from "express";2import * as inferenceProxy from "@inferencesh/sdk/proxy/express";3import rateLimit from "express-rate-limit";4 5const app = express();6app.use(express.json());7 8// Rate limiting9const limiter = rateLimit({10  windowMs: 15 * 60 * 1000, // 15 minutes11  max: 100, // limit each IP to 100 requests per window12});13 14// Auth middleware15const authMiddleware = (req, res, next) => {16  const token = req.headers.authorization;17  if (!validateToken(token)) {18    return res.status(401).json({ error: "Unauthorized" });19  }20  next();21};22 23// Apply middleware before proxy24app.all(25  inferenceProxy.route,26  limiter,27  authMiddleware,28  inferenceProxy.handler29);

CORS Support

typescript
1import cors from "cors";2 3app.use(cors({4  origin: "https://myapp.com",5  methods: ["GET", "POST", "PUT"],6  allowedHeaders: ["Content-Type", "x-inf-target-url"],7}));8 9app.all(inferenceProxy.route, inferenceProxy.handler);

Streaming

The Express handler automatically supports streaming responses for real-time task updates.

we use cookies

we use cookies to ensure you get the best experience on our website. for more information on how we use cookies, please see our cookie policy.

by clicking "accept", you agree to our use of cookies.
learn more.