Exploring The Impact Of Proactive Generative AI Agent Roles In Time-Sensitive Collaborative Problem-Solving Tasks

TL;DR

This study investigates how proactive generative AI agents influence real-time collaborative problem-solving, revealing that peer agents can improve performance but may also cause disruptions, while facilitator agents offer limited support.

Contribution

It provides empirical insights into the effects of proactive AI agents in time-sensitive teamwork, highlighting design considerations for future AI support systems.

Findings

Peer agents can improve problem-solving with timely hints

Peer agents may disrupt flow and increase workload

Facilitator agents offer limited impact on outcomes

Abstract

Collaborative problem-solving under time pressure is common but difficult, as teams must generate ideas quickly, coordinate actions, and track progress. Generative AI offers new opportunities to assist, but we know little about how proactive agents affect the dynamics of real-time, co-located teamwork. We studied two forms of proactive support in digital escape rooms: a facilitator agent that offered summaries and group structures, and a peer agent that proposed ideas and answered queries. In a within-subjects study with 24 participants, we compared group performance and processes across three conditions: no AI, peer, and facilitator. Results show that the peer agent occasionally enhanced problem-solving by offering timely hints and memory support; however, it also disrupted flow, increased workload, and created over-reliance. In comparison, the facilitator agent provided light scaffolding but had a limited impact on outcomes. We provide design considerations for proactive generative AI agents based on our findings.