Teach-DETR: Better Training DETR with Teachers

TL;DR

Teach-DETR introduces a teacher-based training scheme for DETR detectors, leveraging multiple teacher models' predicted boxes to enhance detection accuracy without extra inference cost.

Contribution

The paper proposes a novel training approach that uses teacher detectors' predicted boxes to improve DETR-based models, compatible with multiple teachers and with minimal additional computational cost.

Findings

Improves DINO detector accuracy from 57.8% to 58.9% mAP.

Effective knowledge transfer from RCNN and DETR-based teachers.

No additional inference overhead during deployment.

Abstract

In this paper, we present a novel training scheme, namely Teach-DETR, to learn better DETR-based detectors from versatile teacher detectors. We show that the predicted boxes from teacher detectors are effective medium to transfer knowledge of teacher detectors, which could be either RCNN-based or DETR-based detectors, to train a more accurate and robust DETR model. This new training scheme can easily incorporate the predicted boxes from multiple teacher detectors, each of which provides parallel supervisions to the student DETR. Our strategy introduces no additional parameters and adds negligible computational cost to the original detector during training. During inference, Teach-DETR brings zero additional overhead and maintains the merit of requiring no non-maximum suppression. Extensive experiments show that our method leads to consistent improvement for various DETR-based detectors. Specifically, we improve the state-of-the-art detector DINO with Swin-Large backbone, 4 scales of feature maps and 36-epoch training schedule, from 57.8% to 58.9% in terms of mean average precision on MSCOCO 2017 validation set. Code will be available at https://github.com/LeonHLJ/Teach-DETR.