Multimodal Model with Text and Drug Embeddings for Adverse Drug Reaction Classification

TL;DR

This paper introduces a multimodal model combining text and drug embeddings to improve adverse drug reaction classification from social media tweets, achieving state-of-the-art results across multiple languages.

Contribution

It presents a novel multimodal approach integrating BERT-based language models with molecular property prediction for ADE classification, outperforming previous methods.

Findings

Achieved state-of-the-art F1 scores of 0.61 and 0.57 on #SMM4H 2021 tasks in English and Russian.

Improved F1 by 8% on French tweets compared to previous best.

Neural network-derived molecular information is more effective than traditional descriptors.

Abstract

In this paper, we focus on the classification of tweets as sources of potential signals for adverse drug effects (ADEs) or drug reactions (ADRs). Following the intuition that text and drug structure representations are complementary, we introduce a multimodal model with two components. These components are state-of-the-art BERT-based models for language understanding and molecular property prediction. Experiments were carried out on multilingual benchmarks of the Social Media Mining for Health Research and Applications (#SMM4H) initiative. Our models obtained state-of-the-art results of 0.61 F1 and 0.57 F1 on #SMM4H 2021 Shared Tasks 1a and 2 in English and Russian, respectively. On the classification of French tweets from SMM4H 2020 Task 1, our approach pushes the state of the art by an absolute gain of 8% F1. Our experiments show that the molecular information obtained from neural networks is more beneficial for ADE classification than traditional molecular descriptors. The source code for our models is freely available at https://github.com/Andoree/smm4h_2021_classification.