granite 4.0 h small FP8

Name: granite 4.0 h small FP8
Author: IBM

by IBM

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released October 2025131K context≈ 98,304 words

A compact, efficiently quantized model from IBM's Granite line, running in FP8 precision to reduce memory footprint while preserving capability. It handles a generous 128K token context window, making it capable of processing long documents despite its small size. The FP8 quantization means slightly reduced numerical precision compared to full-precision variants — a deliberate trade-off favoring deployment efficiency.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Long Context

Moderate

Instruction Following

granite 4.0 h small FP8

by IBM

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released October 2025131K context≈ 98,304 words

A compact, efficiently quantized model from IBM's Granite line, running in FP8 precision to reduce memory footprint while preserving capability. It handles a generous 128K token context window, making it capable of processing long documents despite its small size. The FP8 quantization means slightly reduced numerical precision compared to full-precision variants — a deliberate trade-off favoring deployment efficiency.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Long Context

Moderate

Instruction Following

Glossary

Context WindowThe maximum number of tokens a model can process in a single conversation or prompt.FP8 PrecisionA data format that stores numbers using 8 bits instead of the standard 32 bits, significantly reducing memory requirements with minimal quality loss.FP8 QuantizationA compression technique that reduces model size by representing weights using 8-bit floating-point numbers instead of higher precision, making it faster and more memory-efficient.Full-PrecisionA model using standard 32-bit floating-point numbers to represent weights, providing maximum accuracy but requiring more memory.Memory FootprintThe amount of RAM or storage space a model requires to run, which is critical for deployment on resource-constrained devices.PrecisionThe level of numerical detail a model uses to represent its internal values; higher precision means more accurate calculations but requires more memory.QuantizationReducing a model's numerical precision (e.g., from 16-bit to 4-bit) to shrink memory usage and speed up inference.QuantizedA technique that reduces a model's size and memory usage by storing weights with lower precision (fewer bits), trading some accuracy for efficiency.TokenA small unit of text (a word, subword, or punctuation mark) that a language model breaks input into for processing.

Capabilities

Capabilities

Use Case Fit

Glossary