Humanlike Multi-user Agent (HUMA): Designing a Deceptively Human AI Facilitator for Group Chats

TL;DR

This paper introduces HUMA, an LLM-based multi-user chat facilitator that mimics human interaction patterns, making AI agents indistinguishable from humans in group chat settings and maintaining user engagement.

Contribution

HUMA extends multi-user chatbot capabilities with an event-driven architecture and realistic timing, improving naturalness and human-likeness in group conversations.

Findings

Participants could not reliably distinguish HUMA from humans.

Subjective experience with HUMA was comparable to human community managers.

HUMA maintained engagement and social presence similar to human facilitators.

Abstract

Conversational agents built on large language models (LLMs) are becoming increasingly prevalent, yet most systems are designed for one-on-one, turn-based exchanges rather than natural, asynchronous group chats. As AI assistants become widespread throughout digital platforms, from virtual assistants to customer service, developing natural and humanlike interaction patterns seems crucial for maintaining user trust and engagement. We present the Humanlike Multi-user Agent (HUMA), an LLM-based facilitator that participates in multi-party conversations using human-like strategies and timing. HUMA extends prior multi-user chatbot work with an event-driven architecture that handles messages, replies, reactions and introduces realistic response-time simulation. HUMA comprises three components-Router, Action Agent, and Reflection-which together adapt LLMs to group conversation dynamics. We evaluate HUMA in a controlled study with 97 participants in four-person role-play chats, comparing AI and human community managers (CMs). Participants classified CMs as human at near-chance rates in both conditions, indicating they could not reliably distinguish HUMA agents from humans. Subjective experience was comparable across conditions: community-manager effectiveness, social presence, and engagement/satisfaction differed only modestly with small effect sizes. Our results suggest that, in natural group chat settings, an AI facilitator can match human quality while remaining difficult to identify as nonhuman.