Dataset distillation doesn't consistently outperform simpler coreset selection methods on real-world tasks, and coresets are often more computationally efficient for creating compact training datasets.
This paper challenges the effectiveness of dataset distillation by comparing it against simpler coreset selection methods across large-scale benchmarks.