LLMs can learn to abstain from answering when uncertain by explicitly assessing information requirements first—this reduces confident wrong answers by 32% without requiring model retraining, just better prompting.
This paper introduces Chain-of-Self-Questioning (CoSQ), a prompting technique that makes LLMs decide whether to answer or abstain based on assessing what information is needed. Testing on TruthfulQA, CoSQ reduces wrong answers from 13.1% to 8.9% while maintaining 87.6% answer coverage, showing that self-assessment helps models avoid confidently stating things they don't actually know.