You can capture complex LLM reasoning in a tiny classifier by training on natural-language explanations, then adapt it to specific contexts at inference time—enabling LLM-quality decisions at production scale.
This paper solves the problem of recommending product upgrades at massive scale by distilling LLM reasoning into a small, fast classifier, then fine-tuning it per product category. Instead of calling an LLM for millions of product pairs, they train a 15.5M-parameter model on LLM-generated explanations, achieving 95% of LLM quality while being 5,000x faster and 10,000x cheaper.