Robustness to Transformations Across Categories: Is Robustness To Transformations Driven by Invariant Neural Representations?

TL;DR

This paper investigates whether invariant neural representations are the primary reason for robustness to transformations in deep neural networks, finding that invariance is not always necessary and depends on the type and extent of training.

Contribution

The study challenges the assumption that invariance is the main driver of robustness, showing it emerges with more categories and varies with transformation type.

Findings

Invariant representations do not always lead to robustness.

Robustness can occur without invariance for certain categories.

Invariance emerges more with increased transformed categories.

Abstract

Deep Convolutional Neural Networks (DCNNs) have demonstrated impressive robustness to recognize objects under transformations (eg. blur or noise) when these transformations are included in the training set. A hypothesis to explain such robustness is that DCNNs develop invariant neural representations that remain unaltered when the image is transformed. However, to what extent this hypothesis holds true is an outstanding question, as robustness to transformations could be achieved with properties different from invariance, eg. parts of the network could be specialized to recognize either transformed or non-transformed images. This paper investigates the conditions under which invariant neural representations emerge by leveraging that they facilitate robustness to transformations beyond the training distribution. Concretely, we analyze a training paradigm in which only some object categories are seen transformed during training and evaluate whether the DCNN is robust to transformations across categories not seen transformed. Our results with state-of-the-art DCNNs indicate that invariant neural representations do not always drive robustness to transformations, as networks show robustness for categories seen transformed during training even in the absence of invariant neural representations. Invariance only emerges as the number of transformed categories in the training set is increased. This phenomenon is much more prominent with local transformations such as blurring and high-pass filtering than geometric transformations such as rotation and thinning, which entail changes in the spatial arrangement of the object. Our results contribute to a better understanding of invariant neural representations in deep learning and the conditions under which it spontaneously emerges.