NVIDIA Nemotron 3.5 Lightning 30B A3B W4A16

NVIDIA

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released August 2026262K context≈ 196,608 words

A lean, efficient text model that punches above its weight through a mixture-of-experts architecture — only 3 billion parameters are active at a time despite the 30 billion total, keeping inference costs low. It runs in a quantized W4A16 format, trading a small amount of precision for significantly reduced memory footprint. Its standout feature is an unusually large 262K token context window, allowing it to work across very long documents without truncation.

NVIDIA Nemotron 3.5 Lightning 30B A3B W4A16

NVIDIA

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released August 2026262K context≈ 196,608 words

A lean, efficient text model that punches above its weight through a mixture-of-experts architecture — only 3 billion parameters are active at a time despite the 30 billion total, keeping inference costs low. It runs in a quantized W4A16 format, trading a small amount of precision for significantly reduced memory footprint. Its standout feature is an unusually large 262K token context window, allowing it to work across very long documents without truncation.

Glossary

ArchitectureThe underlying structural design of a neural network that defines how data flows through layers and components.Context WindowThe maximum number of tokens a model can process in a single conversation or prompt.InferenceThe process of running a trained model to generate predictions or outputs from new inputs.Memory FootprintThe amount of RAM or storage space a model requires to run, which is critical for deployment on resource-constrained devices.ParametersThe learned numerical values in a model — more parameters generally means more capacity but higher compute cost.PrecisionThe level of numerical detail a model uses to represent its internal values; higher precision means more accurate calculations but requires more memory.QuantizedA technique that reduces a model's size and memory usage by storing weights with lower precision (fewer bits), trading some accuracy for efficiency.Text ModelA language model that processes and generates only text, without support for images, audio, or other media types.TokenA small unit of text (a word, subword, or punctuation mark) that a language model breaks input into for processing.W4A16A quantization format where model weights are stored in 4-bit precision while calculations use 16-bit precision, balancing efficiency with accuracy.