Sarvam releases Prism for fixed-choice decisions Sarvam releases Prism for fixed-choice decisions

Scoring options

Prism builds on Sarvam’s 105B model with a decision head that produces option probabilities in one forward pass. Its training combines cross-entropy with five-percent label smoothing and a Brier loss, which penalizes errors in the probabilities themselves. In backbone attention, options see the shared context and their own contents, but not competing options. Sarvam measured a 44 ms median across 1,000 sequential requests on one B200 after loading, not a hosted latency guarantee.

Self-hosting uses a dedicated loader

The released model card specifies about 120 GB of free memory on an FP8-capable NVIDIA GPU, plus Python 3.10 or newer and a C compiler for Triton kernels. Its custom FP8 weight layout cannot be loaded through AutoModelForCausalLM or general-purpose inference servers. Developers instead use the repository’s prism.py loader. Local requests accept text or JSON state and a map of questions, processed independently in batches of sixteen by default. A choice result returns the selected key and probabilities. A yes-or-no result supplies the probability of yes. An ordered score returns both the expected level index and the most likely level, which need not be identical. The local context limit is 131,072 tokens per question. The embeddings and normalization layers remain in bfloat16; the mixture-of-experts router retains float32 rather than applying FP8 throughout the model. The card reports loading takes about two minutes on a B200; CUDA graph capture at startup can be disabled.

The hosted confidence field has a different meaning

The hosted service has a smaller per-question context ceiling. It does not truncate oversized inputs: one question exceeding the limit causes the entire request to fail with a 422 response. Requests also have a sixty-second timeout, after which no questions are answered and a 503 error is returned. Its choice response separates option probabilities from a rescaled confidence value. With three options, a top probability of 0.80 becomes confidence of 0.70, because the latter measures distance above uniform chance. The documentation says calibration findings apply to the top probability, not that rescaled field. Teams porting a threshold should therefore check which value their application reads. Hosted inference is synchronous and accepts text or JSON, without image input or streamed output. These are separate deployment contracts, despite using the same model name. Hosted choice questions accept 2–255 options; ordered scores accept 2–10 levels.