What does it mean to understand a neural network?

TL;DR

This paper argues that understanding neural networks and brains is more feasible by focusing on their learning and development processes rather than their complex resulting properties.

Contribution

It introduces the idea that developmental and learning rules are simpler to understand than the network's final properties, guiding neuroscience research.

Findings

Neural networks are easier to understand through their code than their learned properties.

Focusing on learning and development may provide better insights into brain function.

The analogy between neural networks and brains suggests new research directions.

Abstract

We can define a neural network that can learn to recognize objects in less than 100 lines of code. However, after training, it is characterized by millions of weights that contain the knowledge about many object types across visual scenes. Such networks are thus dramatically easier to understand in terms of the code that makes them than the resulting properties, such as tuning or connections. In analogy, we conjecture that rules for development and learning in brains may be far easier to understand than their resulting properties. The analogy suggests that neuroscience would benefit from a focus on learning and development.