Barttender: An approachable & interpretable way to compare medical imaging and non-imaging data

TL;DR

Barttender is an interpretable deep learning framework that enables direct comparison of imaging and non-imaging data in healthcare, improving understanding and evaluation of their relative utility for disease prediction.

Contribution

It introduces a novel method to convert non-imaging data into an interpretable format and provides new measures for global feature importance in image-based models.

Findings

Barttender performs comparably to traditional methods on medical datasets.

It offers enhanced explainability for deep learning models in healthcare.

The framework facilitates direct comparison between imaging and non-imaging data.

Abstract

Imaging-based deep learning has transformed healthcare research, yet its clinical adoption remains limited due to challenges in comparing imaging models with traditional non-imaging and tabular data. To bridge this gap, we introduce Barttender, an interpretable framework that uses deep learning for the direct comparison of the utility of imaging versus non-imaging tabular data for tasks like disease prediction. Barttender converts non-imaging tabular features, such as scalar data from electronic health records, into grayscale bars, facilitating an interpretable and scalable deep learning based modeling of both data modalities. Our framework allows researchers to evaluate differences in utility through performance measures, as well as local (sample-level) and global (population-level) explanations. We introduce a novel measure to define global feature importances for image-based deep learning models, which we call gIoU. Experiments on the CheXpert and MIMIC datasets with chest X-rays and scalar data from electronic health records show that Barttender performs comparably to traditional methods and offers enhanced explainability using deep learning models.