PolicyPulse: LLM-Synthesis Tool for Policy Researchers

TL;DR

PolicyPulse is an LLM-based interactive tool that synthesizes online community discussions into organized themes and anecdotes, aiding policy researchers in understanding public opinion and informing policy analysis.

Contribution

This paper introduces PolicyPulse, a novel LLM-powered system that effectively synthesizes public opinions from online discussions for policy research, demonstrating its alignment with authoritative reports and practical utility.

Findings

PolicyPulse's themes matched authoritative policy reports.

Researchers found PolicyPulse useful for sparking research and understanding public perspectives.

Limitations include demographic data gaps and verification challenges.

Abstract

Public opinion shapes policy, yet capturing it effectively to surface diverse perspectives remains challenging. This paper introduces PolicyPulse, an LLM-powered interactive system that synthesizes public experiences from online community discussions to help policy researchers author memos and briefs, leveraging curated real-world anecdotes. Given a specific topic (e.g., "Climate Change"), PolicyPulse returns an organized list of themes (e.g., "Biodiversity Loss" or "Carbon Pricing"), supporting each theme with relevant quotes from real-life anecdotes. We compared PolicyPulse outputs to authoritative policy reports. Additionally, we asked 11 policy researchers across multiple institutions in the Northeastern U.S to compare using PolicyPulse with their expert approach. We found that PolicyPulse's themes aligned with authoritative reports and helped spark research by analyzing existing data, gathering diverse experiences, revealing unexpected themes, and informing survey or interview design. Participants also highlighted limitations including insufficient demographic context and data verification challenges. Our work demonstrates how AI-powered tools can help influence policy-relevant research and shape policy outcomes.