Beyond Skip Connections: Top-Down Modulation for Object Detection

TL;DR

This paper introduces a top-down modulation approach inspired by human vision to enhance object detection by better integrating fine details from lower convolutional layers, leading to improved accuracy on COCO benchmarks.

Contribution

The paper proposes a novel top-down modulation network with lateral connections that effectively incorporate low-level details into object detection models, surpassing traditional skip connection methods.

Findings

Significant AP improvements on COCO testdev benchmark

Effective integration of low-level details enhances detection of hard categories

Outperforms baseline models without additional complex techniques

Abstract

In recent years, we have seen tremendous progress in the field of object detection. Most of the recent improvements have been achieved by targeting deeper feedforward networks. However, many hard object categories such as bottle, remote, etc. require representation of fine details and not just coarse, semantic representations. But most of these fine details are lost in the early convolutional layers. What we need is a way to incorporate finer details from lower layers into the detection architecture. Skip connections have been proposed to combine high-level and low-level features, but we argue that selecting the right features from low-level requires top-down contextual information. Inspired by the human visual pathway, in this paper we propose top-down modulations as a way to incorporate fine details into the detection framework. Our approach supplements the standard bottom-up, feedforward ConvNet with a top-down modulation (TDM) network, connected using lateral connections. These connections are responsible for the modulation of lower layer filters, and the top-down network handles the selection and integration of contextual information and low-level features. The proposed TDM architecture provides a significant boost on the COCO testdev benchmark, achieving 28.6 AP for VGG16, 35.2 AP for ResNet101, and 37.3 for InceptionResNetv2 network, without any bells and whistles (e.g., multi-scale, iterative box refinement, etc.).