Oriented Object Detection with Transformer

TL;DR

This paper introduces O2DETR, a Transformer-based method for oriented object detection that simplifies the process and improves accuracy, achieving state-of-the-art results on the DOTA dataset.

Contribution

The paper presents a novel end-to-end oriented object detection framework using Transformers with a new efficient encoder, significantly reducing computational costs and setting new benchmarks.

Findings

O2DETR achieves up to 3.85 mAP improvement over Faster R-CNN and RetinaNet.

The proposed encoder reduces memory and computational costs.

O2DETR performs competitively on the DOTA dataset.

Abstract

Object detection with Transformers (DETR) has achieved a competitive performance over traditional detectors, such as Faster R-CNN. However, the potential of DETR remains largely unexplored for the more challenging task of arbitrary-oriented object detection problem. We provide the first attempt and implement Oriented Object DEtection with TRansformer ($\bf O^2DETR$) based on an end-to-end network. The contributions of $\rm O^2DETR$ include: 1) we provide a new insight into oriented object detection, by applying Transformer to directly and efficiently localize objects without a tedious process of rotated anchors as in conventional detectors; 2) we design a simple but highly efficient encoder for Transformer by replacing the attention mechanism with depthwise separable convolution, which can significantly reduce the memory and computational cost of using multi-scale features in the original Transformer; 3) our $\rm O^2DETR$ can be another new benchmark in the field of oriented object detection, which achieves up to 3.85 mAP improvement over Faster R-CNN and RetinaNet. We simply fine-tune the head mounted on $\rm O^2DETR$ in a cascaded architecture and achieve a competitive performance over SOTA in the DOTA dataset.