Meta-Learning the Inductive Biases of Simple Neural Circuits

TL;DR

This paper introduces a neural network-based method to infer the implicit inductive biases of neural circuits, aiding understanding of how biological and artificial systems generalize from limited data.

Contribution

It presents a novel meta-learning approach to uncover the inductive biases of neural circuits, applicable to both artificial and biological systems, including spiking neural networks.

Findings

Successfully recovers known biases in linear and kernel regression

Extracts biases from complex supervised learners and neural data

Provides insights into circuit features influencing generalization

Abstract

Training data is always finite, making it unclear how to generalise to unseen situations. But, animals do generalise, wielding Occam's razor to select a parsimonious explanation of their observations. How they do this is called their inductive bias, and it is implicitly built into the operation of animals' neural circuits. This relationship between an observed circuit and its inductive bias is a useful explanatory window for neuroscience, allowing design choices to be understood normatively. However, it is generally very difficult to map circuit structure to inductive bias. Here, we present a neural network tool to bridge this gap. The tool meta-learns the inductive bias by learning functions that a neural circuit finds easy to generalise, since easy-to-generalise functions are exactly those the circuit chooses to explain incomplete data. In systems with analytically known inductive bias, i.e. linear and kernel regression, our tool recovers it. Generally, we show it can flexibly extract inductive biases from supervised learners, including spiking neural networks, and show how it could be applied to real animals. Finally, we use our tool to interpret recent connectomic data illustrating its intended use: understanding the role of circuit features through the resulting inductive bias.