Exploring Learners' Expectations and Engagement When Collaborating with Constructively Controversial Peer Agents

TL;DR

This study investigates how constructively controversial peer agents, guided by CC theory, affect learner engagement and perceptions in online collaborative learning, highlighting the influence of agent behavior regulation and transparency.

Contribution

It explores the application of CC principles to LLM-based peer agents, examining how regulation and transparency impact learner interactions and perceptions.

Findings

Learners valuing control prefer unregulated agents that cease push-back.

Learners valuing intellectual challenge favor regulated agents for creativity.

Transparency reduces perceived agent abilities.

Abstract

Peer agents can supplement real-time collaborative learning in asynchronous online courses. Constructive Controversy (CC) theory suggests that humans deepen their understanding of a topic by confronting and resolving controversies. This study explores whether CC's benefits apply to LLM-based peer agents, focusing on the impact of agents' disputatious behaviors and disclosure of agents' behavior designs on the learning process. In our mixed-method study (n=144), we compare LLMs that follow detailed CC guidelines (regulated) to those guided by broader goals (unregulated) and examine the effects of disclosing the agents' design to users (transparent vs. opaque). Findings show that learners' values influence their agent interaction: those valuing control appreciate unregulated agents' willingness to cease push-back upon request, while those valuing intellectual challenges favor regulated agents for stimulating creativity. Additionally, design transparency lowers learners' perception of agents' abilities. Our findings lay the foundation for designing effective collaborative peer agents in isolated educational settings.