Generalized Shape Metrics on Neural Representations

TL;DR

This paper introduces a comprehensive framework for quantifying and analyzing neural representations across biological and artificial networks, enabling better understanding of their underlying principles and relationships.

Contribution

It develops a rigorous family of metrics for neural representations, modifies existing similarity measures, and introduces new tools for embedding and analyzing neural data.

Findings

New metrics satisfy triangle inequality and respect convolutional biases.

Identified relationships between neural representations and anatomical features.

Demonstrated methods on biological and deep learning datasets.

Abstract

Understanding the operation of biological and artificial networks remains a difficult and important challenge. To identify general principles, researchers are increasingly interested in surveying large collections of networks that are trained on, or biologically adapted to, similar tasks. A standardized set of analysis tools is now needed to identify how network-level covariates -- such as architecture, anatomical brain region, and model organism -- impact neural representations (hidden layer activations). Here, we provide a rigorous foundation for these analyses by defining a broad family of metric spaces that quantify representational dissimilarity. Using this framework we modify existing representational similarity measures based on canonical correlation analysis to satisfy the triangle inequality, formulate a novel metric that respects the inductive biases in convolutional layers, and identify approximate Euclidean embeddings that enable network representations to be incorporated into essentially any off-the-shelf machine learning method. We demonstrate these methods on large-scale datasets from biology (Allen Institute Brain Observatory) and deep learning (NAS-Bench-101). In doing so, we identify relationships between neural representations that are interpretable in terms of anatomical features and model performance.