DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR

TL;DR

This paper introduces a novel query formulation for DETR using dynamic anchor boxes, improving training convergence and detection accuracy by explicitly incorporating box coordinates as queries.

Contribution

It proposes a new query design with dynamic anchor boxes that enhances DETR's performance and training efficiency, providing deeper understanding of query roles in DETR.

Findings

Achieves 45.7% AP on MS-COCO with ResNet50-DC5 in 50 epochs

Improves training convergence and detection accuracy

Validates the effectiveness through extensive experiments

Abstract

We present in this paper a novel query formulation using dynamic anchor boxes for DETR (DEtection TRansformer) and offer a deeper understanding of the role of queries in DETR. This new formulation directly uses box coordinates as queries in Transformer decoders and dynamically updates them layer-by-layer. Using box coordinates not only helps using explicit positional priors to improve the query-to-feature similarity and eliminate the slow training convergence issue in DETR, but also allows us to modulate the positional attention map using the box width and height information. Such a design makes it clear that queries in DETR can be implemented as performing soft ROI pooling layer-by-layer in a cascade manner. As a result, it leads to the best performance on MS-COCO benchmark among the DETR-like detection models under the same setting, e.g., AP 45.7\% using ResNet50-DC5 as backbone trained in 50 epochs. We also conducted extensive experiments to confirm our analysis and verify the effectiveness of our methods. Code is available at \url{https://github.com/SlongLiu/DAB-DETR}.