Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Coding

TL;DR

This paper introduces a knowledge-enhanced prompt-based fine-tuning approach for multi-label ICD coding, significantly improving performance especially on rare disease codes in long medical notes.

Contribution

It proposes a novel knowledge-injected Longformer model with domain-specific knowledge and prompt-based fine-tuning for improved few-shot multi-label ICD coding.

Findings

Outperforms state-of-the-art by 14.5% in macro F1 on MIMIC-III-full

Significantly improves macro F1 from 17.1 to 30.4 on rare disease dataset

Demonstrates effectiveness in long-tail and few-shot ICD coding scenarios

Abstract

Automatic International Classification of Diseases (ICD) coding aims to assign multiple ICD codes to a medical note with average length of 3,000+ tokens. This task is challenging due to a high-dimensional space of multi-label assignment (tens of thousands of ICD codes) and the long-tail challenge: only a few codes (common diseases) are frequently assigned while most codes (rare diseases) are infrequently assigned. This study addresses the long-tail challenge by adapting a prompt-based fine-tuning technique with label semantics, which has been shown to be effective under few-shot setting. To further enhance the performance in medical domain, we propose a knowledge-enhanced longformer by injecting three domain-specific knowledge: hierarchy, synonym, and abbreviation with additional pretraining using contrastive learning. Experiments on MIMIC-III-full, a benchmark dataset of code assignment, show that our proposed method outperforms previous state-of-the-art method in 14.5% in marco F1 (from 10.3 to 11.8, P<0.001). To further test our model on few-shot setting, we created a new rare diseases coding dataset, MIMIC-III-rare50, on which our model improves marco F1 from 17.1 to 30.4 and micro F1 from 17.2 to 32.6 compared to previous method.