Knowledge distillation works better for semantic-ID recommenders when you match distributions either by reconstructing items from codes or by projecting item probabilities to individual code positions—both approaches beat standard hard-label training.
This paper tackles a key challenge in generative recommendation systems: how to transfer knowledge from traditional recommenders to newer semantic-ID models that predict items as sequences of discrete codes. The authors propose two knowledge-distillation methods that bridge the gap between item-level and code-level predictions, achieving significant improvements in recommendation quality.