Machine learning performance has hard mathematical limits set by data structure, not just algorithm choice. Understanding these limits requires proper models of how data is generated, especially for systems with feedback like LLM agents.
This paper examines fundamental limits on what machine learning systems can achieve, showing that performance is constrained by the structure of the data itself rather than just algorithm design.