Emergence of functional and structural properties of the head direction system by optimization of recurrent neural networks

TL;DR

This study demonstrates that optimizing recurrent neural networks for head direction estimation naturally leads to the emergence of neural and structural properties similar to biological head direction circuits in rodents and flies.

Contribution

It shows that unconstrained, goal-driven training of RNNs can reproduce both neural representations and anatomical features of biological head direction systems.

Findings

Emergence of compass and shifter neurons in trained RNNs

Structural similarities between artificial networks and biological circuits

RNNs can model neural activity and anatomy of head direction systems

Abstract

Recent work suggests goal-driven training of neural networks can be used to model neural activity in the brain. While response properties of neurons in artificial neural networks bear similarities to those in the brain, the network architectures are often constrained to be different. Here we ask if a neural network can recover both neural representations and, if the architecture is unconstrained and optimized, the anatomical properties of neural circuits. We demonstrate this in a system where the connectivity and the functional organization have been characterized, namely, the head direction circuits of the rodent and fruit fly. We trained recurrent neural networks (RNNs) to estimate head direction through integration of angular velocity. We found that the two distinct classes of neurons observed in the head direction system, the Compass neurons and the Shifter neurons, emerged naturally in artificial neural networks as a result of training. Furthermore, connectivity analysis and in-silico neurophysiology revealed structural and mechanistic similarities between artificial networks and the head direction system. Overall, our results show that optimization of RNNs in a goal-driven task can recapitulate the structure and function of biological circuits, suggesting that artificial neural networks can be used to study the brain at the level of both neural activity and anatomical organization.