Efficient DETR: Improving End-to-End Object Detector with Dense Prior

TL;DR

Efficient DETR introduces a streamlined end-to-end object detection method that uses dense priors for initialization, reducing the need for multiple decoder layers while maintaining high performance and robustness in crowded scenes.

Contribution

The paper proposes a novel initialization strategy using dense priors, significantly reducing decoder layers needed in DETR without sacrificing accuracy.

Findings

Achieves competitive results with only 3 encoder and 1 decoder layer.

Outperforms state-of-the-art detectors on MS COCO.

Demonstrates robustness in crowded scenes like CrowdHuman.

Abstract

The recently proposed end-to-end transformer detectors, such as DETR and Deformable DETR, have a cascade structure of stacking 6 decoder layers to update object queries iteratively, without which their performance degrades seriously. In this paper, we investigate that the random initialization of object containers, which include object queries and reference points, is mainly responsible for the requirement of multiple iterations. Based on our findings, we propose Efficient DETR, a simple and efficient pipeline for end-to-end object detection. By taking advantage of both dense detection and sparse set detection, Efficient DETR leverages dense prior to initialize the object containers and brings the gap of the 1-decoder structure and 6-decoder structure. Experiments conducted on MS COCO show that our method, with only 3 encoder layers and 1 decoder layer, achieves competitive performance with state-of-the-art object detection methods. Efficient DETR is also robust in crowded scenes. It outperforms modern detectors on CrowdHuman dataset by a large margin.