Pulse of the Pandemic: Iterative Topic Filtering for Clinical Information Extraction from Social Media

TL;DR

This paper introduces an unsupervised, iterative method for extracting clinically relevant information from social media data during health emergencies, enabling rapid identification of emerging topics without manual labeling.

Contribution

The authors develop a novel unsupervised approach combining heuristic filtering, topic modeling, and concept extraction to identify relevant clinical content from large-scale social media data.

Findings

Successfully extracted relevant clinical topics from 52 million tweets.

Identified emerging COVID-19 topics on a weekly basis.

Facilitated knowledge transfer among healthcare professionals.

Abstract

The rapid evolution of the COVID-19 pandemic has underscored the need to quickly disseminate the latest clinical knowledge during a public-health emergency. One surprisingly effective platform for healthcare professionals (HCPs) to share knowledge and experiences from the front lines has been social media (for example, the "#medtwitter" community on Twitter). However, identifying clinically-relevant content in social media without manual labeling is a challenge because of the sheer volume of irrelevant data. We present an unsupervised, iterative approach to mine clinically relevant information from social media data, which begins by heuristically filtering for HCP-authored texts and incorporates topic modeling and concept extraction with MetaMap. This approach identifies granular topics and tweets with high clinical relevance from a set of about 52 million COVID-19-related tweets from January to mid-June 2020. We also show that because the technique does not require manual labeling, it can be used to identify emerging topics on a week-to-week basis. Our method can aid in future public-health emergencies by facilitating knowledge transfer among healthcare workers in a rapidly-changing information environment, and by providing an efficient and unsupervised way of highlighting potential areas for clinical research.