Global Pooling, More than Meets the Eye: Position Information is Encoded Channel-Wise in CNNs

TL;DR

This paper reveals that global pooling in CNNs encodes positional information in channel order, impacting translation invariance and enabling targeted attacks, thus deepening understanding of CNN internal representations.

Contribution

It demonstrates that spatial information persists in global pooled features and introduces methods to analyze and manipulate position encoding in CNNs.

Findings

Positional information is encoded in channel order after global pooling.

Semantic information is largely unaffected by spatial dimension collapsing.

Region-specific attacks can degrade CNN performance in targeted input areas.

Abstract

In this paper, we challenge the common assumption that collapsing the spatial dimensions of a 3D (spatial-channel) tensor in a convolutional neural network (CNN) into a vector via global pooling removes all spatial information. Specifically, we demonstrate that positional information is encoded based on the ordering of the channel dimensions, while semantic information is largely not. Following this demonstration, we show the real world impact of these findings by applying them to two applications. First, we propose a simple yet effective data augmentation strategy and loss function which improves the translation invariance of a CNN's output. Second, we propose a method to efficiently determine which channels in the latent representation are responsible for (i) encoding overall position information or (ii) region-specific positions. We first show that semantic segmentation has a significant reliance on the overall position channels to make predictions. We then show for the first time that it is possible to perform a `region-specific' attack, and degrade a network's performance in a particular part of the input. We believe our findings and demonstrated applications will benefit research areas concerned with understanding the characteristics of CNNs.