Category-orthogonal object features guide information processing in recurrent neural networks trained for object categorization

TL;DR

This study investigates how recurrent neural networks use auxiliary information like location and scale to improve object categorization, especially in cluttered images, revealing that recurrence helps communicate and utilize this information for better performance.

Contribution

It demonstrates that recurrent neural networks transmit and leverage category-orthogonal auxiliary variables to enhance object categorization in challenging visual conditions.

Findings

Auxiliary variable information increases over time in RNN layers.

Recurrent information flow contains auxiliary variable data.

Manipulating auxiliary information impacts categorization accuracy.

Abstract

Recurrent neural networks (RNNs) have been shown to perform better than feedforward architectures in visual object categorization tasks, especially in challenging conditions such as cluttered images. However, little is known about the exact computational role of recurrent information flow in these conditions. Here we test RNNs trained for object categorization on the hypothesis that recurrence iteratively aids object categorization via the communication of category-orthogonal auxiliary variables (the location, orientation, and scale of the object). Using diagnostic linear readouts, we find that: (a) information about auxiliary variables increases across time in all network layers, (b) this information is indeed present in the recurrent information flow, and (c) its manipulation significantly affects task performance. These observations confirm the hypothesis that category-orthogonal auxiliary variable information is conveyed through recurrent connectivity and is used to optimize category inference in cluttered environments.