Predicting Clinical Intent from Free Text Electronic Health Records

TL;DR

This study develops a BERT-based machine learning model to accurately detect clinicians' follow-up intents from electronic health record notes, addressing the challenge of missing intent documentation.

Contribution

The paper introduces a novel approach to identify clinical intent from free-text notes using BERT, with a focus on handling class imbalance and annotating multiple intent types.

Findings

Achieved macro-F1 score of 0.90 in intent classification.

Annotated 3000 clinical notes with 22 intent types, identifying class imbalance.

Limited training data for some intent classes due to imbalance.

Abstract

After a patient consultation, a clinician determines the steps in the management of the patient. A clinician may for example request to see the patient again or refer them to a specialist. Whilst most clinicians will record their intent as "next steps" in the patient's clinical notes, in some cases the clinician may forget to indicate their intent as an order or request, e.g. failure to place the follow-up order. This consequently results in patients becoming lost-to-follow up and may in some cases lead to adverse consequences. In this paper we train a machine learning model to detect a clinician's intent to follow up with a patient from the patient's clinical notes. Annotators systematically identified 22 possible types of clinical intent and annotated 3000 Bariatric clinical notes. The annotation process revealed a class imbalance in the labeled data and we found that there was only sufficient labeled data to train 11 out of the 22 intents. We used the data to train a BERT based multilabel classification model and reported the following average accuracy metrics for all intents: macro-precision: 0.91, macro-recall: 0.90, macro-f1: 0.90.