PHEE: A Dataset for Pharmacovigilance Event Extraction from Text

TL;DR

This paper introduces PHEE, the largest publicly available dataset for pharmacovigilance event extraction, enabling improved automation in drug safety monitoring and analysis of adverse drug reactions from biomedical texts.

Contribution

It provides a novel, large-scale annotated dataset with a hierarchical schema for extracting pharmacovigilance events from medical reports and literature.

Findings

Current state-of-the-art models have limitations on this dataset.

The dataset reveals challenges in biomedical event extraction.

Open challenges for future research are discussed.

Abstract

The primary goal of drug safety researchers and regulators is to promptly identify adverse drug reactions. Doing so may in turn prevent or reduce the harm to patients and ultimately improve public health. Evaluating and monitoring drug safety (i.e., pharmacovigilance) involves analyzing an ever growing collection of spontaneous reports from health professionals, physicians, and pharmacists, and information voluntarily submitted by patients. In this scenario, facilitating analysis of such reports via automation has the potential to rapidly identify safety signals. Unfortunately, public resources for developing natural language models for this task are scant. We present PHEE, a novel dataset for pharmacovigilance comprising over 5000 annotated events from medical case reports and biomedical literature, making it the largest such public dataset to date. We describe the hierarchical event schema designed to provide coarse and fine-grained information about patients' demographics, treatments and (side) effects. Along with the discussion of the dataset, we present a thorough experimental evaluation of current state-of-the-art approaches for biomedical event extraction, point out their limitations, and highlight open challenges to foster future research in this area.