Interactive imitation learning relaxes what learners need to represent: instead of copying the expert's policy exactly, they only need to match the expert's value function, which is often easier when the learner is less expressive than the expert.
This paper investigates when interactive expert queries help in imitation learning. The key insight is that on-policy interaction allows agents to learn by matching the expert's value function rather than their full policy—a weaker requirement. The authors introduce OVI, an algorithm that leverages this insight and prove interaction is necessary without additional assumptions.