Characterising representation dynamics in recurrent neural networks for object recognition

TL;DR

This study investigates the evolving internal representations of recurrent neural networks during object recognition, revealing that representations continue to change after classification and differ between correct and incorrect predictions.

Contribution

It provides new insights into the dynamic behavior of RNN representations in visual tasks and introduces an analysis framework applicable to various RNN architectures.

Findings

Representations evolve after correct classification.

Misclassified representations have lower L2 norm and are more peripheral.

Dynamics are consistent across networks with different connection types.

Abstract

Recurrent neural networks (RNNs) have yielded promising results for both recognizing objects in challenging conditions and modeling aspects of primate vision. However, the representational dynamics of recurrent computations remain poorly understood, especially in large-scale visual models. Here, we studied such dynamics in RNNs trained for object classification on MiniEcoset, a novel subset of ecoset. We report two main insights. First, upon inference, representations continued to evolve after correct classification, suggesting a lack of the notion of being ``done with classification''. Second, focusing on ``readout zones'' as a way to characterize the activation trajectories, we observe that misclassified representations exhibit activation patterns with lower L2 norm, and are positioned more peripherally in the readout zones. Such arrangements help the misclassified representations move into the correct zones as time progresses. Our findings generalize to networks with lateral and top-down connections, and include both additive and multiplicative interactions with the bottom-up sweep. The results therefore contribute to a general understanding of RNN dynamics in naturalistic tasks. We hope that the analysis framework will aid future investigations of other types of RNNs, including understanding of representational dynamics in primate vision.