Bridging the Gap: Leveraging Retrieval-Augmented Generation to Better Understand Public Concerns about Vaccines

TL;DR

This paper introduces VaxPulse Query Corner, a retrieval-augmented generation tool that effectively analyzes social media data to understand public vaccine concerns, improving public health responses.

Contribution

It presents a novel retrieval-augmented generation approach tailored for public health, addressing LLM limitations in capturing current events and nuanced opinions.

Findings

Achieved answer faithfulness of 0.96

Achieved answer relevance of 0.94

Analyzed 35,103 social media posts

Abstract

Vaccine hesitancy threatens public health, leading to delayed or rejected vaccines. Social media is a vital source for understanding public concerns, and traditional methods like topic modelling often struggle to capture nuanced opinions. Though trained for query answering, large Language Models (LLMs) often miss current events and community concerns. Additionally, hallucinations in LLMs can compromise public health communication. To address these limitations, we developed a tool (VaxPulse Query Corner) using the Retrieval Augmented Generation technique. It addresses complex queries about public vaccine concerns on various online platforms, aiding public health administrators and stakeholders in understanding public concerns and implementing targeted interventions to boost vaccine confidence. Analysing 35,103 Shingrix social media posts, it achieved answer faithfulness (0.96) and relevance (0.94).