Understanding the Effective Receptive Field in Deep Convolutional Neural Networks

TL;DR

This paper investigates the effective receptive field in deep convolutional neural networks, revealing it is smaller than the theoretical size and influenced by architecture and nonlinearities, with implications for improving network design.

Contribution

It introduces the concept of an effective receptive field, analyzes its properties across architectures, and explores factors affecting its size, providing insights for better network design.

Findings

Effective receptive field has a Gaussian distribution.

It occupies only a fraction of the theoretical receptive field.

Factors like nonlinearities and skip connections influence its size.

Abstract

We study characteristics of receptive fields of units in deep convolutional networks. The receptive field size is a crucial issue in many visual tasks, as the output must respond to large enough areas in the image to capture information about large objects. We introduce the notion of an effective receptive field, and show that it both has a Gaussian distribution and only occupies a fraction of the full theoretical receptive field. We analyze the effective receptive field in several architecture designs, and the effect of nonlinear activations, dropout, sub-sampling and skip connections on it. This leads to suggestions for ways to address its tendency to be too small.