That's the Wrong Lung! Evaluating and Improving the Interpretability of Unsupervised Multimodal Encoders for Medical Data

TL;DR

This paper evaluates the interpretability of multimodal models for medical data, revealing that current attention-based alignments often do not reflect true anatomical regions and proposing simple methods to improve interpretability.

Contribution

It critically assesses the alignment quality of state-of-the-art multimodal models in medical imaging and introduces minimal supervision techniques to enhance interpretability.

Findings

Attention heatmaps often do not reflect anatomical regions

Synthetic modifications do not significantly change attention highlights

Few-shot finetuning improves alignment quality

Abstract

Pretraining multimodal models on Electronic Health Records (EHRs) provides a means of learning representations that can transfer to downstream tasks with minimal supervision. Recent multimodal models induce soft local alignments between image regions and sentences. This is of particular interest in the medical domain, where alignments might highlight regions in an image relevant to specific phenomena described in free-text. While past work has suggested that attention "heatmaps" can be interpreted in this manner, there has been little evaluation of such alignments. We compare alignments from a state-of-the-art multimodal (image and text) model for EHR with human annotations that link image regions to sentences. Our main finding is that the text has an often weak or unintuitive influence on attention; alignments do not consistently reflect basic anatomical information. Moreover, synthetic modifications -- such as substituting "left" for "right" -- do not substantially influence highlights. Simple techniques such as allowing the model to opt out of attending to the image and few-shot finetuning show promise in terms of their ability to improve alignments with very little or no supervision. We make our code and checkpoints open-source.