Mobile image models need different evaluation criteria than server models—this benchmark shows that expanding task coverage matters more than further optimizing deployment efficiency, and specific capability gaps (like text replacement) should be prioritized over broad improvements.
MoI-Bench is a benchmark for evaluating image generation and editing models on mobile devices, with 1150 samples across 64 subtasks defined by combinations of 16 operations (like removal, replacement) and 8 visual targets (portraits, text, etc.).