Evaluating Representational Similarity Measures from the Lens of Functional Correspondence

TL;DR

This paper evaluates various representational similarity measures in neuroscience and AI, finding that geometry-focused metrics like CKA and Procrustes excel at aligning with behavioral data, guiding better metric selection.

Contribution

It systematically compares eight similarity metrics in the visual domain, highlighting which best capture behavioral relevance and model distinctions.

Findings

CKA and Procrustes outperform others in behavioral alignment

Geometry-based metrics better differentiate trained vs. untrained models

Linear predictivity shows moderate behavioral correlation

Abstract

Neuroscience and artificial intelligence (AI) both face the challenge of interpreting high-dimensional neural data, where the comparative analysis of such data is crucial for revealing shared mechanisms and differences between these complex systems. Despite the widespread use of representational comparisons and the abundance classes of comparison methods, a critical question remains: which metrics are most suitable for these comparisons? While some studies evaluate metrics based on their ability to differentiate models of different origins or constructions (e.g., various architectures), another approach is to assess how well they distinguish models that exhibit distinct behaviors. To investigate this, we examine the degree of alignment between various representational similarity measures and behavioral outcomes, employing group statistics and a comprehensive suite of behavioral metrics for comparison. In our evaluation of eight commonly used representational similarity metrics in the visual domain -- spanning alignment-based, Canonical Correlation Analysis (CCA)-based, inner product kernel-based, and nearest-neighbor methods -- we found that metrics like linear Centered Kernel Alignment (CKA) and Procrustes distance, which emphasize the overall geometric structure or shape of representations, excelled in differentiating trained from untrained models and aligning with behavioral measures, whereas metrics such as linear predictivity, commonly used in neuroscience, demonstrated only moderate alignment with behavior. These insights are crucial for selecting metrics that emphasize behaviorally meaningful comparisons in NeuroAI research.