General Instance Distillation for Object Detection

TL;DR

This paper introduces a novel general instance distillation method for object detection that leverages relation and feature knowledge, significantly improving lightweight model performance across various frameworks.

Contribution

It proposes a general instance distillation approach that does not rely on ground truth labels and effectively utilizes relation-based knowledge for better detection accuracy.

Findings

Student models outperform teachers in various detection frameworks.

GID improves mAP of RetinaNet with ResNet-50 from 36.2% to 39.1%.

Student models can surpass teacher models in detection performance.

Abstract

In recent years, knowledge distillation has been proved to be an effective solution for model compression. This approach can make lightweight student models acquire the knowledge extracted from cumbersome teacher models. However, previous distillation methods of detection have weak generalization for different detection frameworks and rely heavily on ground truth (GT), ignoring the valuable relation information between instances. Thus, we propose a novel distillation method for detection tasks based on discriminative instances without considering the positive or negative distinguished by GT, which is called general instance distillation (GID). Our approach contains a general instance selection module (GISM) to make full use of feature-based, relation-based and response-based knowledge for distillation. Extensive results demonstrate that the student model achieves significant AP improvement and even outperforms the teacher in various detection frameworks. Specifically, RetinaNet with ResNet-50 achieves 39.1% in mAP with GID on COCO dataset, which surpasses the baseline 36.2% by 2.9%, and even better than the ResNet-101 based teacher model with 38.1% AP.