Characterizing Information Seeking Events in Health-Related Social Discourse

TL;DR

This paper introduces a novel event-based framework and dataset for analyzing health-related information-seeking behavior on social media, specifically focusing on opioid use disorder discussions on Reddit.

Contribution

It defines domain-specific event categories, creates the TREAT-ISE dataset, and benchmarks machine learning models for classifying health-related social discourse events.

Findings

Achieved 77.4% F1 score with classifiers

First to define event categories for OUD social discourse

Insights into ChatGPT's performance and errors

Abstract

Social media sites have become a popular platform for individuals to seek and share health information. Despite the progress in natural language processing for social media mining, a gap remains in analyzing health-related texts on social discourse in the context of events. Event-driven analysis can offer insights into different facets of healthcare at an individual and collective level, including treatment options, misconceptions, knowledge gaps, etc. This paper presents a paradigm to characterize health-related information-seeking in social discourse through the lens of events. Events here are board categories defined with domain experts that capture the trajectory of the treatment/medication. To illustrate the value of this approach, we analyze Reddit posts regarding medications for Opioid Use Disorder (OUD), a critical global health concern. To the best of our knowledge, this is the first attempt to define event categories for characterizing information-seeking in OUD social discourse. Guided by domain experts, we develop TREAT-ISE, a novel multilabel treatment information-seeking event dataset to analyze online discourse on an event-based framework. This dataset contains Reddit posts on information-seeking events related to recovery from OUD, where each post is annotated based on the type of events. We also establish a strong performance benchmark (77.4% F1 score) for the task by employing several machine learning and deep learning classifiers. Finally, we thoroughly investigate the performance and errors of ChatGPT on this task, providing valuable insights into the LLM's capabilities and ongoing characterization efforts.