Exploring a Unified Sequence-To-Sequence Transformer for Medical Product Safety Monitoring in Social Media

TL;DR

This paper presents a unified sequence-to-sequence transformer model for detecting and extracting adverse events from social media posts, significantly improving performance and robustness over existing methods, with some cross-language transfer capabilities.

Contribution

It introduces a novel multi-task training strategy for AE detection and extraction using the T5 model, enhancing performance and robustness across tasks and languages.

Findings

Achieved 12.7% relative improvement in AE detection F1 score.

Demonstrated increased robustness with multi-task training.

Showed zero-shot transfer to French social media data.

Abstract

Adverse Events (AE) are harmful events resulting from the use of medical products. Although social media may be crucial for early AE detection, the sheer scale of this data makes it logistically intractable to analyze using human agents, with NLP representing the only low-cost and scalable alternative. In this paper, we frame AE Detection and Extraction as a sequence-to-sequence problem using the T5 model architecture and achieve strong performance improvements over competitive baselines on several English benchmarks (F1 = 0.71, 12.7% relative improvement for AE Detection; Strict F1 = 0.713, 12.4% relative improvement for AE Extraction). Motivated by the strong commonalities between AE-related tasks, the class imbalance in AE benchmarks and the linguistic and structural variety typical of social media posts, we propose a new strategy for multi-task training that accounts, at the same time, for task and dataset characteristics. Our multi-task approach increases model robustness, leading to further performance gains. Finally, our framework shows some language transfer capabilities, obtaining higher performance than Multilingual BERT in zero-shot learning on French data.