What Does CNN Shift Invariance Look Like? A Visualization Study

TL;DR

This study visualizes and measures the shift invariance of CNN features, revealing that popular models lack global invariance and that anti-aliasing improves local but not global invariance, with implications for model robustness.

Contribution

The paper provides a comprehensive visualization and analysis of CNN shift invariance, highlighting biases and the effects of anti-aliasing on invariance properties.

Findings

CNN features are not globally invariant.

Anti-aliased models improve local invariance.

Biases and artifacts affect feature invariance.

Abstract

Feature extraction with convolutional neural networks (CNNs) is a popular method to represent images for machine learning tasks. These representations seek to capture global image content, and ideally should be independent of geometric transformations. We focus on measuring and visualizing the shift invariance of extracted features from popular off-the-shelf CNN models. We present the results of three experiments comparing representations of millions of images with exhaustively shifted objects, examining both local invariance (within a few pixels) and global invariance (across the image frame). We conclude that features extracted from popular networks are not globally invariant, and that biases and artifacts exist within this variance. Additionally, we determine that anti-aliased models significantly improve local invariance but do not impact global invariance. Finally, we provide a code repository for experiment reproduction, as well as a website to interact with our results at https://jakehlee.github.io/visualize-invariance.