
Decision 2.0 Lux 9B
The 9B Decision 2.0 model from vLLM Semantic Router. Send a state and typed questions (choice, score, yes/no); get a probability for every answer in one forward pass, with no text generation. 7.94B parameters, 16,384-token input, about 18.4 ms per question.
api reference
about
the 9b decision 2.0 model from vllm semantic router. send a state and typed questions (choice, score, yes/no); get a probability for every answer in one forward pass, with no text generation. 7.94b parameters, 16,384-token input, about 18.4 ms per question.
get an api key
create one in settings → api keys. the examples read it from INFERENCE_API_KEY.
1export INFERENCE_API_KEY="inf_your_key"1. calling the api
install the client
nothing to install. any http client works; the examples use curl, and jq to read responses. every response wraps its result in a data field.
run and get result
send "wait": true to hold the connection until the task finishes. without it the call returns right away with the task still running.
12# "wait": true holds the connection until the task finishes.3# the response is {"data": {"id", "status", "status_text", "output", ...}}4curl -X POST https://api.inference.sh/apps/run \5 -H "Authorization: Bearer $INFERENCE_API_KEY" \6 -H "Content-Type: application/json" \7 -d '{8 "app": "vllm-sr/decision-2-0-lux-9b",9 "input": {10 "state": "The order arrived damaged yesterday. The customer has a receipt and asks for a replacement today.",11 "choices": [12 {13 "id": "choices_1",14 "instructions": "instructions",15 "options": [16 {17 "name": "options_1"18 },19 {20 "name": "options_2"21 }22 ]23 }24 ],25 "scores": [26 {27 "id": "scores_1",28 "instructions": "instructions",29 "levels": [30 "levels_1",31 "levels_2"32 ]33 }34 ],35 "nouls": [36 {37 "id": "nouls_1",38 "instructions": "instructions"39 }40 ]41 },42 "wait": true43 }'stream live updates
get real-time progress updates as the task runs. ideal for showing progress bars, partial results, or long-running tasks. send "stream": true and the response is a server-sent event stream that ends when the task does.
12# "stream": true answers with server-sent events until the task finishes3curl -N -X POST https://api.inference.sh/apps/run \4 -H "Authorization: Bearer $INFERENCE_API_KEY" \5 -H "Content-Type: application/json" \6 -d '{7 "app": "vllm-sr/decision-2-0-lux-9b",8 "input": {9 "state": "The order arrived damaged yesterday. The customer has a receipt and asks for a replacement today.",10 "choices": [11 {12 "id": "choices_1",13 "instructions": "instructions",14 "options": [15 {16 "name": "options_1"17 },18 {19 "name": "options_2"20 }21 ]22 }23 ],24 "scores": [25 {26 "id": "scores_1",27 "instructions": "instructions",28 "levels": [29 "levels_1",30 "levels_2"31 ]32 }33 ],34 "nouls": [35 {36 "id": "nouls_1",37 "instructions": "instructions"38 }39 ]40 },41 "stream": true42 }'run in the background
start the task and get its id back right away, then check on it later. for long tasks, or when you'd rather be called back, see webhooks.
1# without "wait" the call returns the task right away (status 1, "received")2TASK_ID=$(curl -s -X POST https://api.inference.sh/apps/run \3 -H "Authorization: Bearer $INFERENCE_API_KEY" \4 -H "Content-Type: application/json" \5 -d '{6 "app": "vllm-sr/decision-2-0-lux-9b",7 "input": {8 "state": "The order arrived damaged yesterday. The customer has a receipt and asks for a replacement today.",9 "choices": [10 {11 "id": "choices_1",12 "instructions": "instructions",13 "options": [14 {15 "name": "options_1"16 },17 {18 "name": "options_2"19 }20 ]21 }22 ],23 "scores": [24 {25 "id": "scores_1",26 "instructions": "instructions",27 "levels": [28 "levels_1",29 "levels_2"30 ]31 }32 ],33 "nouls": [34 {35 "id": "nouls_1",36 "instructions": "instructions"37 }38 ]39 }40 }' | jq -r '.data.id')4142# later: read its state. done when status is 10 (completed), 11 (failed) or 12 (cancelled)43curl -s https://api.inference.sh/tasks/$TASK_ID \44 -H "Authorization: Bearer $INFERENCE_API_KEY" | jq '.data | {status, status_text, output}'2. authentication
the api uses api keys for authentication. see the authentication docs for detailed setup instructions.
3. files
file inputs are automatically handled by the sdk. you can pass local paths, urls, or base64 data.
automatic upload
pass urls directly or use data uris for inline content.
1# pass URLs or data URIs as input values2curl -X POST https://api.inference.sh/apps/run \3 -H "Authorization: Bearer $INFERENCE_API_KEY" \4 -H "Content-Type: application/json" \5 -d '{6 "app": "vllm-sr/decision-2-0-lux-9b",7 "input": {8 "image": "https://example.com/image.png",9 "audio": "data:audio/mp3;base64,..."10 },11 "wait": true12 }'manual upload
you can also upload files manually and use the returned url.
1# 1. create the file: the response has an upload_url and the file's uri2FILE=$(curl -s -X POST https://api.inference.sh/files \3 -H "Authorization: Bearer $INFERENCE_API_KEY" \4 -H "Content-Type: application/json" \5 -d '{"files": [{"filename": "image.png", "content_type": "image/png"}]}')67# 2. PUT the bytes to upload_url8curl -X PUT "$(echo "$FILE" | jq -r '.data[0].upload_url')" \9 -H "Content-Type: image/png" \10 --data-binary @image.png1112# 3. pass the uri as an input value13echo "$FILE" | jq -r '.data[0].uri'4. webhooks
get notified when a task completes by providing a webhook url. when the task reaches a terminal state (completed, failed, or cancelled), a POST request is sent to your url with the task result.
1curl -X POST https://api.inference.sh/apps/run \2 -H "Authorization: Bearer $INFERENCE_API_KEY" \3 -H "Content-Type: application/json" \4 -d '{5 "app": "vllm-sr/decision-2-0-lux-9b",6 "input": {7 "state": "The order arrived damaged yesterday. The customer has a receipt and asks for a replacement today.",8 "choices": [9 {10 "id": "choices_1",11 "instructions": "instructions",12 "options": [13 {14 "name": "options_1"15 },16 {17 "name": "options_2"18 }19 ]20 }21 ],22 "scores": [23 {24 "id": "scores_1",25 "instructions": "instructions",26 "levels": [27 "levels_1",28 "levels_2"29 ]30 }31 ],32 "nouls": [33 {34 "id": "nouls_1",35 "instructions": "instructions"36 }37 ]38 },39 "webhook": "https://your-server.com/webhook"40 }'webhook payload
your endpoint receives a JSON POST with the task result:
1{2 "event": "task.completed",3 "timestamp": "2024-01-15T10:30:05Z",4 "data": {5 "id": "task_abc123",6 "short_id": "abc123",7 "status": 10,8 "status_text": "completed",9 "output": { ... },10 "created_at": "2024-01-15T10:30:00Z",11 "updated_at": "2024-01-15T10:30:05Z"12 }13}5. schema
input
the content to evaluate: a string, or a json object / array of related context (messages, records, a policy). text only. every question sees the same state. input over the model's token limit is rejected, never truncated.
choice questions: pick one option from a set.
score questions: place the state on ordered levels.
noul questions: probability that the answer is yes.
new models

Seedream 5.5
bytedance/seedream-5-5
image$0.020 per 1K/1.5K image, $0.035 per 2K image, plus $0.008 per reference image
Kling Video V4 Flash
klingai/video-v4-flash
video$0.094/s at 720p
Kling Video V4
klingai/video-v4
video$0.157/s at 720p, $0.235/s at 1080p
MiniMax H3 Image
minimax/h3-image
image$2.30/M text in, $1.50/M image in, $3.80/M image out (token-based)
Pika H3 Flex (self-hosted)
pika/pika-h3-flex-oss
videoGPU time, billed per millisecond
ready to run Decision 2.0 Lux 9B?
we use cookies
we use cookies to ensure you get the best experience on our website. for more information on how we use cookies, please see our cookie policy.
by clicking "accept", you agree to our use of cookies.
learn more.