D2Q-DETR: Decoupling and Dynamic Queries for Oriented Object Detection with Transformers

TL;DR

D2Q-DETR introduces a transformer-based, end-to-end oriented object detection framework that decouples query features, employs dynamic queries, and improves label assignment, achieving state-of-the-art results on large aerial image datasets.

Contribution

The paper proposes a novel DETR-based framework with decoupled query features, dynamic query design, and improved label re-assignment for oriented object detection.

Findings

Outperforms existing methods on DOTA datasets

Reduces object queries without performance loss

Achieves state-of-the-art accuracy in aerial image detection

Abstract

Despite the promising results, existing oriented object detection methods usually involve heuristically designed rules, e.g., RRoI generation, rotated NMS. In this paper, we propose an end-to-end framework for oriented object detection, which simplifies the model pipeline and obtains superior performance. Our framework is based on DETR, with the box regression head replaced with a points prediction head. The learning of points is more flexible, and the distribution of points can reflect the angle and size of the target rotated box. We further propose to decouple the query features into classification and regression features, which significantly improves the model precision. Aerial images usually contain thousands of instances. To better balance model precision and efficiency, we propose a novel dynamic query design, which reduces the number of object queries in stacked decoder layers without sacrificing model performance. Finally, we rethink the label assignment strategy of existing DETR-like detectors and propose an effective label re-assignment strategy for improved performance. We name our method D2Q-DETR. Experiments on the largest and challenging DOTA-v1.0 and DOTA-v1.5 datasets show that D2Q-DETR outperforms existing NMS-based and NMS-free oriented object detection methods and achieves the new state-of-the-art.