When deploying pruned LLMs for tool use, mixture-of-experts architectures are significantly more robust than dense models, and you need to evaluate specific action components—not just overall accuracy—to catch degradation early.
This paper studies how pruning (removing parts of neural networks to reduce size) affects large language models' ability to control smart home devices. The researchers test four different LLMs with various pruning strategies, finding that dense models break suddenly after modest pruning, while mixture-of-experts models are more robust.