TransICD: Transformer Based Code-wise Attention Model for Explainable ICD Coding

TL;DR

TransICD introduces a transformer-based model with code-wise attention for automated ICD coding, improving accuracy and interpretability in medical note classification, and addressing dataset imbalance with LDAM loss.

Contribution

The paper proposes a novel transformer architecture with code-wise attention and LDAM loss for improved ICD code prediction and interpretability on clinical datasets.

Findings

Achieves a micro-AUC of 0.923 on MIMIC-III dataset.

Outperforms baseline models like bidirectional RNNs significantly.

Provides interpretable insights through code-wise attention mechanism.

Abstract

International Classification of Disease (ICD) coding procedure which refers to tagging medical notes with diagnosis codes has been shown to be effective and crucial to the billing system in medical sector. Currently, ICD codes are assigned to a clinical note manually which is likely to cause many errors. Moreover, training skilled coders also requires time and human resources. Therefore, automating the ICD code determination process is an important task. With the advancement of artificial intelligence theory and computational hardware, machine learning approach has emerged as a suitable solution to automate this process. In this project, we apply a transformer-based architecture to capture the interdependence among the tokens of a document and then use a code-wise attention mechanism to learn code-specific representations of the entire document. Finally, they are fed to separate dense layers for corresponding code prediction. Furthermore, to handle the imbalance in the code frequency of clinical datasets, we employ a label distribution aware margin (LDAM) loss function. The experimental results on the MIMIC-III dataset show that our proposed model outperforms other baselines by a significant margin. In particular, our best setting achieves a micro-AUC score of 0.923 compared to 0.868 of bidirectional recurrent neural networks. We also show that by using the code-wise attention mechanism, the model can provide more insights about its prediction, and thus it can support clinicians to make reliable decisions. Our code is available online (https://github.com/biplob1ly/TransICD)