LP-DETR: Layer-wise Progressive Relations for Object Detection

TL;DR

LP-DETR introduces a layer-wise progressive relation modeling approach that enhances object detection by adaptively learning multi-scale spatial relations, resulting in faster convergence and higher accuracy on COCO dataset.

Contribution

The paper proposes a novel layer-wise progressive relation mechanism for DETR, improving multi-scale relation modeling and detection performance.

Findings

Achieves 52.3% AP with 12 epochs on COCO

Learns to prioritize local relations early and global relations later

Improves convergence speed and detection accuracy

Abstract

This paper presents LP-DETR (Layer-wise Progressive DETR), a novel approach that enhances DETR-based object detection through multi-scale relation modeling. Our method introduces learnable spatial relationships between object queries through a relation-aware self-attention mechanism, which adaptively learns to balance different scales of relations (local, medium and global) across decoder layers. This progressive design enables the model to effectively capture evolving spatial dependencies throughout the detection pipeline. Extensive experiments on COCO 2017 dataset demonstrate that our method improves both convergence speed and detection accuracy compared to standard self-attention module. The proposed method achieves competitive results, reaching 52.3\% AP with 12 epochs and 52.5\% AP with 24 epochs using ResNet-50 backbone, and further improving to 58.0\% AP with Swin-L backbone. Furthermore, our analysis reveals an interesting pattern: the model naturally learns to prioritize local spatial relations in early decoder layers while gradually shifting attention to broader contexts in deeper layers, providing valuable insights for future research in object detection.