# Modelling Temporal Document Sequences for Clinical ICD Coding

## TL;DR

This paper introduces a hierarchical transformer model that leverages all clinical notes during hospital stays, significantly improving ICD coding accuracy over models that only use discharge summaries.

## Contribution

The paper presents a novel hierarchical transformer architecture that incorporates all clinical notes and metadata for improved ICD coding, reducing training costs with superconvergence.

## Key findings

- Model surpasses state-of-the-art with discharge summaries alone.
- Using all clinical notes further improves coding performance.
- Superconvergence reduces training time significantly.

## Abstract

Past studies on the ICD coding problem focus on predicting clinical codes primarily based on the discharge summary. This covers only a small fraction of the notes generated during each hospital stay and leaves potential for improving performance by analysing all the available clinical notes. We propose a hierarchical transformer architecture that uses text across the entire sequence of clinical notes in each hospital stay for ICD coding, and incorporates embeddings for text metadata such as their position, time, and type of note. While using all clinical notes increases the quantity of data substantially, superconvergence can be used to reduce training costs. We evaluate the model on the MIMIC-III dataset. Our model exceeds the prior state-of-the-art when using only discharge summaries as input, and achieves further performance improvements when all clinical notes are used as input.