You can build AI systems that automatically route uncertain decisions to humans for review, with mathematical guarantees that a user-defined fraction of errors won't be missed—and do this more efficiently than checking everything or checking randomly.
This paper tackles the problem of when to trust AI decisions versus when to ask a human expert to review them. The authors develop algorithms that automatically decide whether to accept an AI system's output or escalate it for human auditing, with guaranteed bounds on how many errors slip through unreviewed.