Conformal prediction set sizes have a rigorous information-theoretic interpretation: they quantify information gain in a way that's mathematically sandwiched between generalized entropy measures and obeys data processing inequalities.
This paper establishes a theoretical connection between conformal prediction (a method for uncertainty quantification) and information theory. The authors show that the size of prediction sets from conformal methods can be interpreted as a measure of information gain, providing mathematical justification for using set size as an uncertainty metric.