Self-consistency certification methods used to verify LLM correctness fail under realistic correlation in reasoning samples; you need prompt-level aggregation, not within-prompt resampling, to make valid population-level reliability claims.
This paper reveals a critical flaw in self-consistency certification for language models: when reasoning samples are correlated (which they are in practice), standard anytime-valid tests certify the wrong thing.