Concept-based model explanations for Electronic Health Records

TL;DR

This paper extends Concept Activation Vectors (TCAV) for use with sequential time series data in electronic health records, enabling interpretable deep learning models in clinical settings.

Contribution

We adapt TCAV for sequential EHR data, allowing human-understandable explanations of RNN predictions in healthcare applications.

Findings

Successfully applied TCAV extension to ICU EHR data

Demonstrated interpretability on synthetic and real-world datasets

Enhanced trust in deep models for clinical decision-making

Abstract

Recurrent Neural Networks (RNNs) are often used for sequential modeling of adverse outcomes in electronic health records (EHRs) due to their ability to encode past clinical states. These deep, recurrent architectures have displayed increased performance compared to other modeling approaches in a number of tasks, fueling the interest in deploying deep models in clinical settings. One of the key elements in ensuring safe model deployment and building user trust is model explainability. Testing with Concept Activation Vectors (TCAV) has recently been introduced as a way of providing human-understandable explanations by comparing high-level concepts to the network's gradients. While the technique has shown promising results in real-world imaging applications, it has not been applied to structured temporal inputs. To enable an application of TCAV to sequential predictions in the EHR, we propose an extension of the method to time series data. We evaluate the proposed approach on an open EHR benchmark from the intensive care unit, as well as synthetic data where we are able to better isolate individual effects.