Node Specificity in Convolutional Deep Nets Depends on Receptive Field Position and Size

TL;DR

This paper investigates how the specificity of receptive fields in convolutional neural networks varies with position and size, revealing that large RFs can lead to position-dependent features, challenging the traditional view of invariance.

Contribution

It demonstrates that receptive field size and position influence feature specificity, highlighting a nuanced understanding of spatial information processing in deep nets.

Findings

Receptive field size increases with network depth.

Large receptive fields cause position-dependent feature specificity.

Challenging the assumption of strict translational invariance in CNNs.

Abstract

In convolutional deep neural networks, receptive field (RF) size increases with hierarchical depth. When RF size approaches full coverage of the input image, different RF positions result in RFs with different specificity, as portions of the RF fall out of the input space. This leads to a departure from the convolutional concept of positional invariance and opens the possibility for complex forms of context specificity.