Deep Representational Similarity Learning for analyzing neural signatures in task-based fMRI dataset

TL;DR

This paper introduces Deep Representational Similarity Learning (DRSL), a neural network-based extension of RSA, designed for analyzing complex neural signatures in large, high-dimensional fMRI datasets across multiple subjects and cognitive tasks.

Contribution

DRSL provides a flexible, nonlinear approach to RSA that can handle high-dimensional data and multiple subjects, outperforming existing linear and kernel-based methods.

Findings

DRSL achieves superior accuracy compared to state-of-the-art RSA methods.

It effectively analyzes diverse cognitive tasks in multi-subject fMRI datasets.

The method reduces runtime through gradient-based optimization.

Abstract

Similarity analysis is one of the crucial steps in most fMRI studies. Representational Similarity Analysis (RSA) can measure similarities of neural signatures generated by different cognitive states. This paper develops Deep Representational Similarity Learning (DRSL), a deep extension of RSA that is appropriate for analyzing similarities between various cognitive tasks in fMRI datasets with a large number of subjects, and high-dimensionality -- such as whole-brain images. Unlike the previous methods, DRSL is not limited by a linear transformation or a restricted fixed nonlinear kernel function -- such as Gaussian kernel. DRSL utilizes a multi-layer neural network for mapping neural responses to linear space, where this network can implement a customized nonlinear transformation for each subject separately. Furthermore, utilizing a gradient-based optimization in DRSL can significantly reduce runtime of analysis on large datasets because it uses a batch of samples in each iteration rather than all neural responses to find an optimal solution. Empirical studies on multi-subject fMRI datasets with various tasks -- including visual stimuli, decision making, flavor, and working memory -- confirm that the proposed method achieves superior performance to other state-of-the-art RSA algorithms.