Partial Soft-Matching Distance for Neural Representational Comparison with Partial Unit Correspondence

TL;DR

This paper introduces a partial soft-matching distance for neural representational comparison that allows some units to remain unmatched, improving robustness and interpretability in noisy or outlier-prone data.

Contribution

It extends soft-matching to a partial optimal transport framework, enabling selective unit matching and efficient neuron ranking for better neural and brain data analysis.

Findings

Preserves correct matches under outliers and noise.

Automatically excludes low-reliability voxels in fMRI data.

Achieves higher alignment precision than standard soft-matching.

Abstract

Representational similarity metrics typically force all units to be matched, making them susceptible to noise and outliers common in neural representations. We extend the soft-matching distance to a partial optimal transport setting that allows some neurons to remain unmatched, yielding rotation-sensitive but robust correspondences. This partial soft-matching distance provides theoretical advantages -- relaxing strict mass conservation while maintaining interpretable transport costs -- and practical benefits through efficient neuron ranking in terms of cross-network alignment without costly iterative recomputation. In simulations, it preserves correct matches under outliers and reliably selects the correct model in noise-corrupted identification tasks. On fMRI data, it automatically excludes low-reliability voxels and produces voxel rankings by alignment quality that closely match computationally expensive brute-force approaches. It achieves higher alignment precision across homologous brain areas than standard soft-matching, which is forced to match all units regardless of quality. In deep networks, highly matched units exhibit similar maximally exciting images, while unmatched units show divergent patterns. This ability to partition by match quality enables focused analyses, e.g., testing whether networks have privileged axes even within their most aligned subpopulations. Overall, partial soft-matching provides a principled and practical method for representational comparison under partial correspondence.