D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement

TL;DR

D-FINE introduces a novel approach to object detection by refining bounding box regression into a distribution refinement process, significantly improving localization accuracy and real-time performance in DETR models.

Contribution

The paper proposes D-FINE, which redefines regression as distribution refinement and introduces self-distillation, achieving state-of-the-art real-time detection accuracy.

Findings

D-FINE-L / X achieves 54.0% / 55.8% AP at 124 / 78 FPS.

Pretraining on Objects365 boosts AP to 57.1% / 59.3%.

Enhances various DETR models by up to 5.3% AP with minimal extra cost.

Abstract

We introduce D-FINE, a powerful real-time object detector that achieves outstanding localization precision by redefining the bounding box regression task in DETR models. D-FINE comprises two key components: Fine-grained Distribution Refinement (FDR) and Global Optimal Localization Self-Distillation (GO-LSD). FDR transforms the regression process from predicting fixed coordinates to iteratively refining probability distributions, providing a fine-grained intermediate representation that significantly enhances localization accuracy. GO-LSD is a bidirectional optimization strategy that transfers localization knowledge from refined distributions to shallower layers through self-distillation, while also simplifying the residual prediction tasks for deeper layers. Additionally, D-FINE incorporates lightweight optimizations in computationally intensive modules and operations, achieving a better balance between speed and accuracy. Specifically, D-FINE-L / X achieves 54.0% / 55.8% AP on the COCO dataset at 124 / 78 FPS on an NVIDIA T4 GPU. When pretrained on Objects365, D-FINE-L / X attains 57.1% / 59.3% AP, surpassing all existing real-time detectors. Furthermore, our method significantly enhances the performance of a wide range of DETR models by up to 5.3% AP with negligible extra parameters and training costs. Our code and pretrained models: https://github.com/Peterande/D-FINE.