Should All Proposals be Treated Equally in Object Detection?

TL;DR

This paper proposes a dynamic proposal processing method that assigns more computation to promising object proposals, improving detection accuracy under resource constraints by learning proposal-specific operators.

Contribution

It introduces a novel end-to-end learnable dynamic routing mechanism for proposals, enhancing efficiency and accuracy in object detection.

Findings

DPP outperforms state-of-the-art detectors at the same computational cost.

Proposal-specific operator assignment improves detection precision.

The method is simple to implement and compatible with existing detectors.

Abstract

The complexity-precision trade-off of an object detector is a critical problem for resource constrained vision tasks. Previous works have emphasized detectors implemented with efficient backbones. The impact on this trade-off of proposal processing by the detection head is investigated in this work. It is hypothesized that improved detection efficiency requires a paradigm shift, towards the unequal processing of proposals, assigning more computation to good proposals than poor ones. This results in better utilization of available computational budget, enabling higher accuracy for the same FLOPS. We formulate this as a learning problem where the goal is to assign operators to proposals, in the detection head, so that the total computational cost is constrained and the precision is maximized. The key finding is that such matching can be learned as a function that maps each proposal embedding into a one-hot code over operators. While this function induces a complex dynamic network routing mechanism, it can be implemented by a simple MLP and learned end-to-end with off-the-shelf object detectors. This 'dynamic proposal processing' (DPP) is shown to outperform state-of-the-art end-to-end object detectors (DETR, Sparse R-CNN) by a clear margin for a given computational complexity.