When training deep learning models on tabular data, use quantile loss instead of MSE for better regression performance, but be aware that global early stopping may compromise performance across regions with different noise levels—consider region-specific stopping or heteroskedasticity-aware obj...
This paper systematically compares regression objectives (loss functions) for deep learning on tabular data, finding that quantile loss consistently outperforms standard MSE. The authors identify a fundamental limitation: different regions of data with varying noise levels reach peak performance at different training epochs, making a single early-stopping point suboptimal.