Multi-agent systems help when communication bandwidth forces useful compression, but hurt when it causes information loss—especially for stronger models that don't need compression. Design multi-agent systems as information bottleneck optimization problems.
This paper explains when multi-agent LLM systems outperform single-agent systems using information theory. The key insight: multi-agent systems compress information through limited communication between agents, creating a trade-off between reducing redundant context (good) and losing task-relevant information (bad).