Actively evaluating and learning the distinctions that matter: Vaccine safety signal detection from emergency triage notes

TL;DR

This paper develops an NLP-based active learning approach to efficiently detect vaccine safety signals from emergency department notes, improving post-vaccination surveillance accuracy and speed.

Contribution

It introduces a novel combination of active learning, data augmentation, and NLP techniques for rapid vaccine safety signal detection from triage notes.

Findings

Enhanced classifier accuracy for vaccine safety signals.

Reduced annotation effort through active learning.

Faster deployment of surveillance models.

Abstract

The rapid development of COVID-19 vaccines has showcased the global communitys ability to combat infectious diseases. However, the need for post-licensure surveillance systems has grown due to the limited window for safety data collection in clinical trials and early widespread implementation. This study aims to employ Natural Language Processing techniques and Active Learning to rapidly develop a classifier that detects potential vaccine safety issues from emergency department notes. ED triage notes, containing expert, succinct vital patient information at the point of entry to health systems, can significantly contribute to timely vaccine safety signal surveillance. While keyword-based classification can be effective, it may yield false positives and demand extensive keyword modifications. This is exacerbated by the infrequency of vaccination-related ED presentations and their similarity to other reasons for ED visits. NLP offers a more accurate and efficient alternative, albeit requiring annotated data, which is often scarce in the medical field. Active learning optimizes the annotation process and the quality of annotated data, which can result in faster model implementation and improved model performance. This work combines active learning, data augmentation, and active learning and evaluation techniques to create a classifier that is used to enhance vaccine safety surveillance from ED triage notes.