Focus Agent: LLM-Powered Virtual Focus Group

TL;DR

This paper presents Focus Agent, an LLM-powered framework that simulates and moderates focus groups, enabling efficient data collection and analysis with comparable quality to human-led sessions, thus reducing resource requirements.

Contribution

The study introduces Focus Agent, a novel LLM-based system that simulates and moderates focus groups, demonstrating comparable opinion quality to human participants and highlighting potential improvements in focus group moderation.

Findings

Focus Agent generates opinions similar to human participants.

LLM moderators improve focus group discussion quality.

Efficient data collection with reduced resource use.

Abstract

In the domain of Human-Computer Interaction, focus groups represent a widely utilised yet resource-intensive methodology, often demanding the expertise of skilled moderators and meticulous preparatory efforts. This study introduces the ``Focus Agent,'' a Large Language Model (LLM) powered framework that simulates both the focus group (for data collection) and acts as a moderator in a focus group setting with human participants. To assess the data quality derived from the Focus Agent, we ran five focus group sessions with a total of 23 human participants as well as deploying the Focus Agent to simulate these discussions with AI participants. Quantitative analysis indicates that Focus Agent can generate opinions similar to those of human participants. Furthermore, the research exposes some improvements associated with LLMs acting as moderators in focus group discussions that include human participants.