HumP-KD: A Hybrid Uncertainty-Aware Multi-Stage Progressive Knowledge Distillation Framework for Efficient Fire Classification

Mohammed Arif Mainuddin, Najifa Tabassum, Omar Ibne Shahid, Riasat Khan|June 12, 2026arXiv

Key Takeaway

You can compress state-of-the-art vision transformers into tiny, CPU-deployable models for real-time fire detection without sacrificing accuracy by using uncertainty-aware, multi-stage knowledge distillation with spatial attention guidance.

Summary

HumP-KD is a knowledge distillation framework that compresses large transformer models (Swin-Tiny, ViT-Base) into a lightweight MobileViT-S student for real-time fire detection.

efficiency training evaluation

Key Terms

knowledge-distillation vision-transformer swin-transformer model-compression attention-mechanism