Studying Limits of Explainability by Integrated Gradients for Gene Expression Models

TL;DR

This paper investigates the limitations of using Integrated Gradients for identifying biomarkers in gene expression models, highlighting challenges in robustness and interpretability through simulations and real data analysis.

Contribution

It introduces a hierarchical simulation model for gene expression data and discusses best practices for evaluating explainability methods in genomics.

Findings

Ranking features by importance alone is insufficient for robust biomarker identification.

Simulated data reveals limitations of Integrated Gradients in capturing true biomarkers.

Proposes guidelines for evaluating explanations in genomics applications.

Abstract

Understanding the molecular processes that drive cellular life is a fundamental question in biological research. Ambitious programs have gathered a number of molecular datasets on large populations. To decipher the complex cellular interactions, recent work has turned to supervised machine learning methods. The scientific questions are formulated as classical learning problems on tabular data or on graphs, e.g. phenotype prediction from gene expression data. In these works, the input features on which the individual predictions are predominantly based are often interpreted as indicative of the cause of the phenotype, such as cancer identification. Here, we propose to explore the relevance of the biomarkers identified by Integrated Gradients, an explainability method for feature attribution in machine learning. Through a motivating example on The Cancer Genome Atlas, we show that ranking features by importance is not enough to robustly identify biomarkers. As it is difficult to evaluate whether biomarkers reflect relevant causes without known ground truth, we simulate gene expression data by proposing a hierarchical model based on Latent Dirichlet Allocation models. We also highlight good practices for evaluating explanations for genomics data and propose a direction to derive more insights from these explanations.