A Pseudo Label-wise Attention Network for Automatic ICD Coding

TL;DR

This paper introduces a pseudo label-wise attention network for automatic ICD coding that reduces computational redundancy by merging similar ICD codes, leading to improved accuracy and the ability to predict new codes.

Contribution

The paper proposes a novel pseudo label-wise attention mechanism that merges similar ICD codes to enhance efficiency and accuracy in multi-label ICD coding tasks.

Findings

Achieves higher micro F1 scores on MIMIC-III and Xiangya datasets.

Effectively predicts new ICD codes using similarity-based inference.

Demonstrates superior performance over existing models.

Abstract

Automatic International Classification of Diseases (ICD) coding is defined as a kind of text multi-label classification problem, which is difficult because the number of labels is very large and the distribution of labels is unbalanced. The label-wise attention mechanism is widely used in automatic ICD coding because it can assign weights to every word in full Electronic Medical Records (EMR) for different ICD codes. However, the label-wise attention mechanism is computational redundant and costly. In this paper, we propose a pseudo label-wise attention mechanism to tackle the problem. Instead of computing different attention modes for different ICD codes, the pseudo label-wise attention mechanism automatically merges similar ICD codes and computes only one attention mode for the similar ICD codes, which greatly compresses the number of attention modes and improves the predicted accuracy. In addition, we apply a more convenient and effective way to obtain the ICD vectors, and thus our model can predict new ICD codes by calculating the similarities between EMR vectors and ICD vectors. Extensive experiments show the superior performance of our model. On the public MIMIC-III dataset and private Xiangya dataset, our model achieves micro f1 of 0.583 and 0.806, respectively, which outperforms other competing models. Furthermore, we verify the ability of our model in predicting new ICD codes. The case study shows how pseudo label-wise attention works, and demonstrates the effectiveness of pseudo label-wise attention mechanism.