Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learning

Ali Larian, Qian Lin, Chang Zong Wu, Daniel S. Brown|July 9, 2026arXiv

Key Takeaway

To learn reward functions that generalize across environments, you need to teach the agent in multiple diverse environments and mix different feedback types—not just collect demonstrations in one setting.

Summary

This paper tackles a key challenge in deploying AI agents: learning reward functions that work across different environments rather than just the one where training happened. The authors show theoretically that different types of human feedback (like comparisons vs.

training alignment evaluation

Key Terms

inverse-reinforcement-learning reward-learning machine-teaching multi-modal-feedback environment-generalization