You can characterize the minimal information needed to hand off an LLM task between sessions using predictive equivalence; for Gaussian regression, this gives exact finite-dimensional representations with quantified memory-accuracy tradeoffs.
When an LLM task continues in a new session (due to context limits, restarts, or agent handoff), the application must decide what information to pass along. This paper develops a theory of 'handover'—transferring task-relevant in-context learning state between sessions—showing what information is sufficient to preserve performance and how much memory it requires.