CRAFT: Clustering Rubrics to Diagnose Weak LLM Capabilities and Generate Targeted Fine-Tuning Data

Vipul Gupta, Zihao Wang, Razvan-Gabriel Dumitru, MohammadHossein Rezaei, Aakash Sabharwal et al.|July 17, 2026arXiv

Key Takeaway

By diagnosing model weaknesses at the granular level of individual rubric criteria rather than whole prompts, you can generate more targeted fine-tuning data that produces measurably better models across multiple domains.

Summary

CRAFT is a diagnostic method that analyzes rubric-based evaluations to identify specific capability gaps in language models, then uses those insights to generate targeted fine-tuning data.

evaluation training data

Key Terms

rubric-generation capability-probe hierarchical-capability-tree targeted-fine-tuning-data supervised-finetuning