Instead of using the same precision everywhere, MixFrag identifies which transformer layers are most fragile to quantization and protects them with higher precision while using lower precision elsewhere—delivering better accuracy under the same memory budget.
MixFrag is a method for compressing Vision Transformers by assigning different precision levels (bit-widths) to different layers based on how sensitive each layer is to quantization. It measures this sensitivity using KL divergence and solves an optimization problem to allocate bits efficiently, achieving strong results on image classification and object detection tasks.