When LLMs handle subjective tasks, explicitly modeling uncertainty and using Bayesian decision theory to choose outputs can improve results, but risk-averse approaches may backfire by favoring generic responses.
This paper develops uncertainty-aware decision-making algorithms for LLMs in subjective tasks like tutoring and peer review. The authors use Bayesian decision theory and conformal prediction to account for model uncertainty when generating outputs, finding that Bayesian approaches work better than risk-averse methods for improving output quality.