Concept Extraction to Identify Adverse Drug Reactions in Medical Forums: A Comparison of Algorithms

TL;DR

This study compares various algorithms for extracting medical concepts from social media posts to improve adverse drug reaction detection, highlighting the impact of method choice on pharmacovigilance effectiveness.

Contribution

It is the first systematic evaluation of concept extraction methods' effects on adverse drug reaction signal detection in social media data.

Findings

Machine learning method outperforms dictionary-based approaches

Choice of algorithm significantly affects concept extraction quality

Proposed method performs well with limited annotated data

Abstract

Social media is becoming an increasingly important source of information to complement traditional pharmacovigilance methods. In order to identify signals of potential adverse drug reactions, it is necessary to first identify medical concepts in the social media text. Most of the existing studies use dictionary-based methods which are not evaluated independently from the overall signal detection task. We compare different approaches to automatically identify and normalise medical concepts in consumer reviews in medical forums. Specifically, we implement several dictionary-based methods popular in the relevant literature, as well as a method we suggest based on a state-of-the-art machine learning method for entity recognition. MetaMap, a popular biomedical concept extraction tool, is used as a baseline. Our evaluations were performed in a controlled setting on a common corpus which is a collection of medical forum posts annotated with concepts and linked to controlled vocabularies such as MedDRA and SNOMED CT. To our knowledge, our study is the first to systematically examine the effect of popular concept extraction methods in the area of signal detection for adverse reactions. We show that the choice of algorithm or controlled vocabulary has a significant impact on concept extraction, which will impact the overall signal detection process. We also show that our proposed machine learning approach significantly outperforms all the other methods in identification of both adverse reactions and drugs, even when trained with a relatively small set of annotated text.