sarvam 30b FP8 dynamic

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released March 2026131K context≈ 98,304 words

A text-focused model quantized to FP8 dynamic precision, which reduces memory footprint while aiming to preserve the quality of the full-precision original. It handles a large 131k-token context window, making it capable of processing lengthy documents in a single pass. The FP8 dynamic quantization means it trades a small amount of numerical precision for significantly faster inference and lower hardware requirements.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Factual Knowledge

Moderate

Instruction Following

sarvam 30b FP8 dynamic

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released March 2026131K context≈ 98,304 words

A text-focused model quantized to FP8 dynamic precision, which reduces memory footprint while aiming to preserve the quality of the full-precision original. It handles a large 131k-token context window, making it capable of processing lengthy documents in a single pass. The FP8 dynamic quantization means it trades a small amount of numerical precision for significantly faster inference and lower hardware requirements.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Factual Knowledge

Moderate

Instruction Following

Glossary

Context WindowThe maximum number of tokens a model can process in a single conversation or prompt.Dynamic QuantizationA quantization approach that adjusts precision levels during inference based on the input data, optimizing the balance between speed and accuracy on-the-fly.FP8 Dynamic QuantizationA compression technique that reduces model size and speeds up inference by representing weights and activations using 8-bit floating-point numbers, with dynamic scaling adjusted per batch to maintain accuracy.FP8 Dynamic QuantizationA specific quantization method that uses 8-bit floating-point numbers and adjusts precision dynamically based on the data being processed, balancing speed and accuracy.Full-PrecisionA model using standard 32-bit floating-point numbers to represent weights, providing maximum accuracy but requiring more memory.InferenceThe process of running a trained model to generate predictions or outputs from new inputs.Memory FootprintThe amount of RAM or storage space a model requires to run, which is critical for deployment on resource-constrained devices.PrecisionThe level of numerical detail a model uses to represent its internal values; higher precision means more accurate calculations but requires more memory.QuantizationReducing a model's numerical precision (e.g., from 16-bit to 4-bit) to shrink memory usage and speed up inference.QuantizedA technique that reduces a model's size and memory usage by storing weights with lower precision (fewer bits), trading some accuracy for efficiency.Text-Focused ModelA model designed specifically to process and generate text, without support for images, audio, or other data types.Text-Focused ModelA language model designed to work exclusively with text input and output, without support for images, audio, or other modalities.TokenA small unit of text (a word, subword, or punctuation mark) that a language model breaks input into for processing.

Capabilities

Capabilities

Use Case Fit

Glossary