apps/vllm-sr

vllm-sr

12 apps available on inference shell.run via API, SDK, or belt CLI.

12apps
1category
Decision 3.0 d3 27B

Decision 3.0 d3 27B

d3

the largest decision 3.0 model from vllm semantic router. send text, images and videos with typed questions (choice, score, yes/no); get a probability for every answer.

decision
Decision 3.0 d3-flash 9B

Decision 3.0 d3-flash 9B

d3-flash

the 9b decision 3.0 model from vllm semantic router. send text, images and videos with typed questions (choice, score, yes/no); get a probability for every answer.

decision
Decision 3.0 d3-mini 4B

Decision 3.0 d3-mini 4B

d3-mini

the 4b decision 3.0 model from vllm semantic router. send text, images and videos with typed questions (choice, score, yes/no); get a probability for every answer.

decision
Decision 3.0 d3-nano 2B

Decision 3.0 d3-nano 2B

d3-nano

the 2b decision 3.0 model from vllm semantic router. send text, images and videos with typed questions (choice, score, yes/no); get a probability for every answer.

decision
Decision 3.0 d3-lite 0.8B

Decision 3.0 d3-lite 0.8B

d3-lite

the 0.8b decision 3.0 model from vllm semantic router. send text, images and videos with typed questions (choice, score, yes/no); get a probability for every answer.

decision
Decision 3.0 d3-edge 0.6B

Decision 3.0 d3-edge 0.6B

d3-edge

the smallest and fastest decision 3.0 model from vllm semantic router. send text, images and videos with typed questions (choice, score, yes/no); get a probability for every answer.

decision
Decision 2.0 Vega 27B

Decision 2.0 Vega 27B

decision-2-0-vega-27b

the largest decision 2.0 model from vllm semantic router. send a state and typed questions (choice, score, yes/no); get a probability for every answer in one pass. 29.37b parameters, 32k-token input.

decision
Decision 2.0 Lux 9B

Decision 2.0 Lux 9B

decision-2-0-lux-9b

the 9b decision 2.0 model from vllm semantic router. send a state and typed questions (choice, score, yes/no); get a probability for every answer in one forward pass, with no text generation. 7.94b parameters, 16,384-token input, about 18.4 ms per question.

decision
Decision 2.0 Nox 4B

Decision 2.0 Nox 4B

decision-2-0-nox-4b

the 4b decision 2.0 model from vllm semantic router. send a state and typed questions (choice, score, yes/no); get a probability for every answer in one forward pass, with no text generation. 4.21b parameters, 16,384-token input, about 12.9 ms per question.

decision
Decision 2.0 Sol 2B

Decision 2.0 Sol 2B

decision-2-0-sol-2b

the 2b decision 2.0 model from vllm semantic router. send a state and typed questions (choice, score, yes/no); get a probability for every answer in one forward pass, with no text generation. 1.88b parameters, 16,384-token input, about 7.2 ms per question.

decision
Decision 2.0 Eos 0.8B

Decision 2.0 Eos 0.8B

decision-2-0-eos-0-8b

a small, fast decision 2.0 model from vllm semantic router. send a state and typed questions (choice, score, yes/no); get a probability for every answer in one forward pass, with no text generation. 0.75b parameters, 16,384-token input, about 6.0 ms per question.

decision
Decision 2.0 Kai 0.6B

Decision 2.0 Kai 0.6B

decision-2-0-kai-0-6b

the smallest and fastest decision 2.0 model from vllm semantic router. send a state and typed questions (choice, score, yes/no); get a probability for every answer in one forward pass, with no text generation. 0.60b parameters, 8,192-token input, about 4.9 ms per question.

decision

explore more on inference shell

vllm-sr is one of many providers on the grid. discover hundreds of apps across image, video, audio, and more.

we use cookies

we use cookies to ensure you get the best experience on our website. for more information on how we use cookies, please see our cookie policy.

by clicking "accept", you agree to our use of cookies.
learn more.