Enhancing Your Trained DETRs with Box Refinement

TL;DR

RefineBox is a lightweight, plugin-based framework that improves DETR-like object detectors by refining their outputs without retraining the entire model, leading to significant performance gains.

Contribution

We introduce RefineBox, a simple and general plugin for enhancing DETR-like models' localization accuracy without retraining the whole detector.

Findings

Performance improvements of 1.6 to 2.5 AP on COCO and LVIS datasets.

RefineBox is easy to implement, train, and generalize across models.

Significant localization accuracy gains demonstrate effectiveness.

Abstract

We present a conceptually simple, efficient, and general framework for localization problems in DETR-like models. We add plugins to well-trained models instead of inefficiently designing new models and training them from scratch. The method, called RefineBox, refines the outputs of DETR-like detectors by lightweight refinement networks. RefineBox is easy to implement and train as it only leverages the features and predicted boxes from the well-trained detection models. Our method is also efficient as we freeze the trained detectors during training. In addition, we can easily generalize RefineBox to various trained detection models without any modification. We conduct experiments on COCO and LVIS $1.0$. Experimental results indicate the effectiveness of our RefineBox for DETR and its representative variants (Figure 1). For example, the performance gains for DETR, Conditinal-DETR, DAB-DETR, and DN-DETR are 2.4 AP, 2.5 AP, 1.9 AP, and 1.6 AP, respectively. We hope our work will bring the attention of the detection community to the localization bottleneck of current DETR-like models and highlight the potential of the RefineBox framework. Code and models will be publicly available at: \href{https://github.com/YiqunChen1999/RefineBox}{https://github.com/YiqunChen1999/RefineBox}.