Is Representational Similarity Analysis Reliable? A Comparison with Regression

TL;DR

This study compares the reliability of Representational Similarity Analysis (RSA) and regression in neuroimaging data analysis, finding regression generally more accurate for model selection and fitting across various conditions.

Contribution

The paper provides a comprehensive evaluation of RSA's reliability compared to regression, highlighting RSA's limitations in model selection accuracy through simulations and empirical data.

Findings

RSA has lower model selection accuracy than regression.

Multicollinearity mitigation improves RSA performance but does not surpass regression.

Regression outperforms RSA in distinguishing between models in empirical and simulated data.

Abstract

Representational Similarity Analysis (RSA) is a popular method for analyzing neuroimaging and behavioral data. Here we evaluate the accuracy and reliability of RSA in the context of model selection, and compare it to that of regression. Although RSA offers flexibility in handling high-dimensional, cross-modal, and cross-species data, its reliance on a transformation of raw data into similarity structures may result in the loss of critical stimulus-response information. Across extensive simulation studies and empirical analyses, we show that RSA leads to lower model selection accuracy, regardless of sample size, noise level, feature dimensionality, or multicollinearity, relative to regression. While principal component analysis and feature reweighting mitigate RSA's deficits driven by multicollinearity, regression remains superior in accurately distinguishing between models. Empirical data and a follow-up fMRI simulation further support these conclusions. Our findings suggest that researchers should carefully consider which approach to use: RSA is less effective than linear regression for model selection and fitting when direct stimulus-response mappings are available.