A single small model can effectively decide when to assist and generate helpful hints by leveraging memory grounding, outperforming larger multi-model systems on speed while maintaining quality—memory context matters far more than retrieval or model size.
This paper tackles proactive assistance in conversations by combining gating (deciding when to help) and hint generation (what to suggest) into a single compact model. The authors show that a 1B parameter decoder can match larger two-model pipelines while being 1.31-1.76x faster, with memory-grounded hints being the key driver of quality.