Using LLMs as judges with careful prompt engineering and ROC-guided model selection can deliver competitive translation quality evaluation at a fraction of typical inference costs.
Pterodactyl is a cost-efficient system for evaluating machine translation quality at the segment level. It uses an LLM-as-a-judge approach with optimized prompting to predict quality scores and identify error-free translations, designed for practical deployment in translation workflows.