Faithful and Plausible Explanations of Medical Code Predictions

TL;DR

This paper introduces a method to generate explanations for medical code predictions that are both faithful to the model and plausible to clinicians, balancing interpretability and trust in high-stakes settings.

Contribution

The authors propose training a proxy model to provide controllable, faithful, and plausible explanations for complex predictive models in medical coding tasks.

Findings

Proxy model explanations are faithful to the original model.

The approach balances explanation faithfulness and plausibility.

Effective in clinical note ICD code prediction.

Abstract

Machine learning models that offer excellent predictive performance often lack the interpretability necessary to support integrated human machine decision-making. In clinical medicine and other high-risk settings, domain experts may be unwilling to trust model predictions without explanations. Work in explainable AI must balance competing objectives along two different axes: 1) Explanations must balance faithfulness to the model's decision-making with their plausibility to a domain expert. 2) Domain experts desire local explanations of individual predictions and global explanations of behavior in aggregate. We propose to train a proxy model that mimics the behavior of the trained model and provides fine-grained control over these trade-offs. We evaluate our approach on the task of assigning ICD codes to clinical notes to demonstrate that explanations from the proxy model are faithful and replicate the trained model behavior.