What Representational Similarity Measures Imply about Decodable Information

TL;DR

This paper links neural representational similarity measures to their implications for decodable information, revealing that many geometric measures reflect the ability to linearly decode stimulus features from neural responses.

Contribution

It demonstrates that popular similarity measures like CKA, CCA, and Procrustes are equivalent to assessing the alignment of optimal linear decoders, providing a new decoding-based interpretation.

Findings

CKA and CCA measure average alignment of linear decoders.

Procrustes distance bounds the difference between optimal decoders.

The work unifies geometric and decoding perspectives on neural representations.

Abstract

Neural responses encode information that is useful for a variety of downstream tasks. A common approach to understand these systems is to build regression models or ``decoders'' that reconstruct features of the stimulus from neural responses. Popular neural network similarity measures like centered kernel alignment (CKA), canonical correlation analysis (CCA), and Procrustes shape distance, do not explicitly leverage this perspective and instead highlight geometric invariances to orthogonal or affine transformations when comparing representations. Here, we show that many of these measures can, in fact, be equivalently motivated from a decoding perspective. Specifically, measures like CKA and CCA quantify the average alignment between optimal linear readouts across a distribution of decoding tasks. We also show that the Procrustes shape distance upper bounds the distance between optimal linear readouts and that the converse holds for representations with low participation ratio. Overall, our work demonstrates a tight link between the geometry of neural representations and the ability to linearly decode information. This perspective suggests new ways of measuring similarity between neural systems and also provides novel, unifying interpretations of existing measures.