Adaptive Instance Distillation for Object Detection in Autonomous Driving

TL;DR

This paper introduces Adaptive Instance Distillation (AID), a novel method for improving object detection models in autonomous driving by selectively transferring knowledge based on instance importance, leading to better performance.

Contribution

The paper proposes AID, a new adaptive knowledge distillation technique that adjusts instance weights based on prediction loss, enhancing detection accuracy in autonomous driving scenarios.

Findings

AID improves mAP by 2.7% for single-stage detectors.

AID enhances mAP by 2.1% for two-stage detectors.

AID benefits self-distillation, boosting teacher model performance.

Abstract

In recent years, knowledge distillation (KD) has been widely used to derive efficient models. Through imitating a large teacher model, a lightweight student model can achieve comparable performance with more efficiency. However, most existing knowledge distillation methods are focused on classification tasks. Only a limited number of studies have applied knowledge distillation to object detection, especially in time-sensitive autonomous driving scenarios. In this paper, we propose Adaptive Instance Distillation (AID) to selectively impart teacher's knowledge to the student to improve the performance of knowledge distillation. Unlike previous KD methods that treat all instances equally, our AID can attentively adjust the distillation weights of instances based on the teacher model's prediction loss. We verified the effectiveness of our AID method through experiments on the KITTI and the COCO traffic datasets. The results show that our method improves the performance of state-of-the-art attention-guided and non-local distillation methods and achieves better distillation results on both single-stage and two-stage detectors. Compared to the baseline, our AID led to an average of 2.7% and 2.1% mAP increases for single-stage and two-stage detectors, respectively. Furthermore, our AID is also shown to be useful for self-distillation to improve the teacher model's performance.