Balanced and Explainable Social Media Analysis for Public Health with Large Language Models

TL;DR

This paper introduces the ALEX framework that combines data augmentation and prompting techniques with large language models to improve social media-based public health analysis, addressing data imbalance and explainability.

Contribution

The paper proposes a novel ALEX framework integrating data augmentation and LLM prompting for better social media public health analysis, achieving top results in SMM4H tasks.

Findings

Achieved first place in two SMM4H tasks

Effective data imbalance resolution through augmentation

Enhanced model explainability via LLM prompting

Abstract

As social media becomes increasingly popular, more and more public health activities emerge, which is worth noting for pandemic monitoring and government decision-making. Current techniques for public health analysis involve popular models such as BERT and large language models (LLMs). Although recent progress in LLMs has shown a strong ability to comprehend knowledge by being fine-tuned on specific domain datasets, the costs of training an in-domain LLM for every specific public health task are especially expensive. Furthermore, such kinds of in-domain datasets from social media are generally highly imbalanced, which will hinder the efficiency of LLMs tuning. To tackle these challenges, the data imbalance issue can be overcome by sophisticated data augmentation methods for social media datasets. In addition, the ability of the LLMs can be effectively utilised by prompting the model properly. In light of the above discussion, in this paper, a novel ALEX framework is proposed for social media analysis on public health. Specifically, an augmentation pipeline is developed to resolve the data imbalance issue. Furthermore, an LLMs explanation mechanism is proposed by prompting an LLM with the predicted results from BERT models. Extensive experiments conducted on three tasks at the Social Media Mining for Health 2023 (SMM4H) competition with the first ranking in two tasks demonstrate the superior performance of the proposed ALEX method. Our code has been released in https://github.com/YanJiangJerry/ALEX.