ICDBigBird: A Contextual Embedding Model for ICD Code Classification

TL;DR

This paper introduces ICDBigBird, a novel model combining BigBird and GCN to improve ICD code classification from large clinical notes, outperforming previous models.

Contribution

The paper presents a new BigBird-based model with GCN integration for better ICD code classification on large clinical documents.

Findings

ICDBigBird outperforms previous state-of-the-art models.

The model effectively processes large clinical notes.

Graph relations between ICD codes enhance embedding quality.

Abstract

The International Classification of Diseases (ICD) system is the international standard for classifying diseases and procedures during a healthcare encounter and is widely used for healthcare reporting and management purposes. Assigning correct codes for clinical procedures is important for clinical, operational, and financial decision-making in healthcare. Contextual word embedding models have achieved state-of-the-art results in multiple NLP tasks. However, these models have yet to achieve state-of-the-art results in the ICD classification task since one of their main disadvantages is that they can only process documents that contain a small number of tokens which is rarely the case with real patient notes. In this paper, we introduce ICDBigBird a BigBird-based model which can integrate a Graph Convolutional Network (GCN), that takes advantage of the relations between ICD codes in order to create 'enriched' representations of their embeddings, with a BigBird contextual model that can process larger documents. Our experiments on a real-world clinical dataset demonstrate the effectiveness of our BigBird-based model on the ICD classification task as it outperforms the previous state-of-the-art models.