Think
LLM
Models
Capabilities
Use Cases
Benchmarks
Papers
Glossary
Search
/
Glossary
/
Inference Overhead
Inference Overhead
techniques
The additional computational cost or latency required to run a model during prediction time.
Inference Overhead — Glossary — ThinkLLM