Comparing interpretation methods in mental state decoding analyses with deep learning models

TL;DR

This paper compares different explanation methods in deep learning models used for decoding mental states from fMRI data, revealing a trade-off between biological plausibility and faithfulness of explanations.

Contribution

It systematically evaluates explanation methods in mental state decoding, highlighting the trade-off between biological plausibility and faithfulness, and offers practical recommendations.

Findings

High faithfulness explanations are less biologically plausible.

Low faithfulness explanations are more biologically plausible.

Guidelines for choosing interpretation methods in mental state decoding.

Abstract

Deep learning (DL) models find increasing application in mental state decoding, where researchers seek to understand the mapping between mental states (e.g., perceiving fear or joy) and brain activity by identifying those brain regions (and networks) whose activity allows to accurately identify (i.e., decode) these states. Once a DL model has been trained to accurately decode a set of mental states, neuroimaging researchers often make use of interpretation methods from explainable artificial intelligence research to understand the model's learned mappings between mental states and brain activity. Here, we compare the explanation performance of prominent interpretation methods in a mental state decoding analysis of three functional Magnetic Resonance Imaging (fMRI) datasets. Our findings demonstrate a gradient between two key characteristics of an explanation in mental state decoding, namely, its biological plausibility and faithfulness: interpretation methods with high explanation faithfulness, which capture the model's decision process well, generally provide explanations that are biologically less plausible than the explanations of interpretation methods with less explanation faithfulness. Based on this finding, we provide specific recommendations for the application of interpretation methods in mental state decoding.