NADE: A Benchmark for Robust Adverse Drug Events Extraction in Face of Negations

TL;DR

This paper evaluates the robustness of ADE extraction models against negation in social media texts and proposes strategies to improve their accuracy, including negation detection and dataset augmentation, with results showing significant performance gains.

Contribution

It introduces a benchmark dataset and evaluates strategies to enhance ADE extraction robustness against negation in social media texts.

Findings

Both proposed strategies significantly improve model performance.

Negation detection reduces false positives in ADE extraction.

Models remain fragile without negation-aware training.

Abstract

Adverse Drug Event (ADE) extraction models can rapidly examine large collections of social media texts, detecting mentions of drug-related adverse reactions and trigger medical investigations. However, despite the recent advances in NLP, it is currently unknown if such models are robust in face of negation, which is pervasive across language varieties. In this paper we evaluate three state-of-the-art systems, showing their fragility against negation, and then we introduce two possible strategies to increase the robustness of these models: a pipeline approach, relying on a specific component for negation detection; an augmentation of an ADE extraction dataset to artificially create negated samples and further train the models. We show that both strategies bring significant increases in performance, lowering the number of spurious entities predicted by the models. Our dataset and code will be publicly released to encourage research on the topic.