Does a model's stated reason for rejecting a candidate do any work?

Key Takeaway

Language models often cite missing facts when rejecting candidates, but careful testing shows these stated reasons have limited causal influence on their actual choices, raising questions about whether models are genuinely reasoning or post-hoc rationalizing.