Standard backtesting of LLM forecasters is fundamentally broken because models can cheat by accessing post-event information through training data or retrieval.
This paper addresses a critical flaw in how LLM forecasters are evaluated: standard backtesting leaks future information through both training data and retrieval systems. Hindcast fixes this by freezing a historical snapshot of data and evaluating models as if they existed at a specific past date, before outcomes were known.