Interpretability of Machine Learning Methods Applied to Neuroimaging

TL;DR

This paper reviews interpretability methods for neural networks in neuroimaging, emphasizing their importance for validating models and detecting biases, and discusses the challenges in choosing and assessing these methods.

Contribution

It provides an overview of key interpretability methods and metrics, focusing on their application, reliability, and benchmarking in neuroimaging, addressing current gaps in the field.

Findings

Most interpretability methods are not yet mature for neuroimaging.

There is a lack of standardized benchmarks for assessing interpretability methods.

Choosing the right interpretability method remains a challenge for practitioners.

Abstract

Deep learning methods have become very popular for the processing of natural images, and were then successfully adapted to the neuroimaging field. As these methods are non-transparent, interpretability methods are needed to validate them and ensure their reliability. Indeed, it has been shown that deep learning models may obtain high performance even when using irrelevant features, by exploiting biases in the training set. Such undesirable situations can potentially be detected by using interpretability methods. Recently, many methods have been proposed to interpret neural networks. However, this domain is not mature yet. Machine learning users face two major issues when aiming to interpret their models: which method to choose, and how to assess its reliability? Here, we aim at providing answers to these questions by presenting the most common interpretability methods and metrics developed to assess their reliability, as well as their applications and benchmarks in the neuroimaging context. Note that this is not an exhaustive survey: we aimed to focus on the studies which we found to be the most representative and relevant.