A separate learned component can better decide when a language model should abstain than the model's own confidence scores, but this approach doesn't generalize reliably across all datasets—a key challenge for deploying models that know their limits.
This paper investigates whether language models can learn when to abstain from answering questions. The researchers compare two approaches: training the model itself versus training a separate component to decide which answers to keep.