Adaptive smoothing in Naive Bayes can reduce prediction errors by up to 22% on high-cardinality data and improve calibration by 41-70% compared to standard fixed-smoothing approaches, with theoretical guarantees that match optimal rates.
This paper improves Naive Bayes classifiers by replacing fixed smoothing rules with a data-adaptive approach that learns smoothing strength from the data itself. The method uses hierarchical empirical Bayes to adjust how much prior information to add based on feature cardinality and sample size, achieving better probability estimates and calibration on real-world datasets.