The Anatomy of Evidence: An Investigation Into Explainable ICD Coding

TL;DR

This paper analyzes the MDACE dataset to evaluate explainable ICD coding systems, revealing partial alignment with ground truth evidence and offering insights for improving explainability in medical coding.

Contribution

It provides an in-depth analysis of the MDACE dataset and evaluates current explainable medical coding methods from an applied perspective.

Findings

Ground truth evidence partially aligns with code descriptions

State-of-the-art approaches show high overlap with ground truth evidence

Recommendations for developing and evaluating explainable coding systems

Abstract

Automatic medical coding has the potential to ease documentation and billing processes. For this task, transparency plays an important role for medical coders and regulatory bodies, which can be achieved using explainability methods. However, the evaluation of these approaches has been mostly limited to short text and binary settings due to a scarcity of annotated data. Recent efforts by Cheng et al. (2023) have introduced the MDACE dataset, which provides a valuable resource containing code evidence in clinical records. In this work, we conduct an in-depth analysis of the MDACE dataset and perform plausibility evaluation of current explainable medical coding systems from an applied perspective. With this, we contribute to a deeper understanding of automatic medical coding and evidence extraction. Our findings reveal that ground truth evidence aligns with code descriptions to a certain degree. An investigation into state-of-the-art approaches shows a high overlap with ground truth evidence. We propose match measures and highlight success and failure cases. Based on our findings, we provide recommendations for developing and evaluating explainable medical coding systems.