Emergent Neural Network Mechanisms for Generalization to Objects in Novel Orientations

TL;DR

This paper investigates how deep neural networks can generalize to recognize objects in novel orientations by disseminating orientation-invariance learned from familiar objects, revealing brain-like neural mechanisms.

Contribution

It demonstrates that DNNs can generalize to new object orientations through feature-based dissemination, especially with more familiar objects and 2D rotations.

Findings

DNNs generalize better with more familiar objects.

Dissemination occurs via neurons tuned to shared features.

Generalization is stronger for 2D rotations.

Abstract

The capability of Deep Neural Networks (DNNs) to recognize objects in orientations outside the distribution of the training data is not well understood. We present evidence that DNNs are capable of generalizing to objects in novel orientations by disseminating orientation-invariance obtained from familiar objects seen from many viewpoints. This capability strengthens when training the DNN with an increasing number of familiar objects, but only in orientations that involve 2D rotations of familiar orientations. We show that this dissemination is achieved via neurons tuned to common features between familiar and unfamiliar objects. These results implicate brain-like neural mechanisms for generalization.