Align-DETR: Enhancing End-to-end Object Detection with Aligned Loss

TL;DR

Align-DETR introduces a novel aligned loss function to address misalignments in DETR, significantly improving convergence and detection accuracy in end-to-end object detection tasks.

Contribution

The paper proposes Align Loss to resolve classification-regression and cross-layer misalignments, enhancing DETR's performance and robustness with a joint quality metric and intermediate layer supervision.

Findings

Achieves 50.5% AP in 1x setting, surpassing previous methods.

Sets new state-of-the-art performance in object detection.

Improves convergence stability and detection accuracy.

Abstract

DETR has set up a simple end-to-end pipeline for object detection by formulating this task as a set prediction problem, showing promising potential. Despite its notable advancements, this paper identifies two key forms of misalignment within the model: classification-regression misalignment and cross-layer target misalignment. Both issues impede DETR's convergence and degrade its overall performance. To tackle both issues simultaneously, we introduce a novel loss function, termed as Align Loss, designed to resolve the discrepancy between the two tasks. Align Loss guides the optimization of DETR through a joint quality metric, strengthening the connection between classification and regression. Furthermore, it incorporates an exponential down-weighting term to facilitate a smooth transition from positive to negative samples. Align-DETR also employs many-to-one matching for supervision of intermediate layers, akin to the design of H-DETR, which enhances robustness against instability. We conducted extensive experiments, yielding highly competitive results. Notably, our method achieves a 49.3% (+0.6) AP on the H-DETR baseline with the ResNet-50 backbone. It also sets a new state-of-the-art performance, reaching 50.5% AP in the 1x setting and 51.7% AP in the 2x setting, surpassing several strong competitors. Our code is available at https://github.com/FelixCaae/AlignDETR.