Search-Aware Reinforcement Learning for Multi-Component Query Understanding in Roblox Game Search

Nayoung Choi, Shengjian Chen, Xiaokai Wei, Wenzheng Zhang, Daiyao Yi et al.|September 24, 2026arXiv

Key Takeaway

Optimizing search query understanding components individually with search-engine-derived rewards outperforms single end-to-end optimization, showing that understanding how each component affects downstream retrieval matters more than just matching labels.

Summary

This paper presents a reinforcement learning framework for query understanding in search systems that optimizes multiple components (like intent classification and query expansion) separately using rewards from live search engine interactions, rather than treating it as a single end-to-end problem.

training reasoning applications

Key Terms

query-understanding reinforcement-learning supervised-fine-tuning component-specific-optimization search-engine-coupled-outputs