A mathematical tool for analyzing the long-term behavior of learning algorithms by averaging over visited states.