A decision model's ability to teach a policy through reinforcement learning is independent of its performance as an actor—poor actors can be excellent judges and teachers because they evaluate actions with full trajectory information.
This paper investigates how decision models—systems that output choices or scores—perform in three different roles: as actors (choosing actions), judges (evaluating past actions), and teachers (training policies via reinforcement learning).