RQFormer: Rotated Query Transformer for End-to-End Oriented Object Detection

TL;DR

RQFormer introduces a novel rotated query transformer with aligned attention and distinct query selection, significantly improving end-to-end oriented object detection accuracy across multiple datasets.

Contribution

The paper proposes RQFormer, a new end-to-end oriented detector combining Rotated RoI Attention and Selective Distinct Queries for better alignment and optimization.

Findings

Effective on remote sensing datasets

Improves one-to-one label assignment

Extends to horizontal object detection

Abstract

Oriented object detection presents a challenging task due to the presence of object instances with multiple orientations, varying scales, and dense distributions. Recently, end-to-end detectors have made significant strides by employing attention mechanisms and refining a fixed number of queries through consecutive decoder layers. However, existing end-to-end oriented object detectors still face two primary challenges: 1) misalignment between positional queries and keys, leading to inconsistency between classification and localization; and 2) the presence of a large number of similar queries, which complicates one-to-one label assignments and optimization. To address these limitations, we propose an end-to-end oriented detector called the Rotated Query Transformer, which integrates two key technologies: Rotated RoI Attention (RRoI Attention) and Selective Distinct Queries (SDQ). First, RRoI Attention aligns positional queries and keys from oriented regions of interest through cross-attention. Second, SDQ collects queries from intermediate decoder layers and filters out similar ones to generate distinct queries, thereby facilitating the optimization of one-to-one label assignments. Finally, extensive experiments conducted on four remote sensing datasets and one scene text dataset demonstrate the effectiveness of our method. To further validate its generalization capability, we also extend our approach to horizontal object detection The code is available at \url{https://github.com/wokaikaixinxin/RQFormer}.