"Can You Tell Me?": Designing Copilots to Support Human Judgement in Online Information Seeking

TL;DR

This study explores an LLM-based conversational copilot designed to support human judgment and digital literacy during online information seeking, revealing deep user engagement but limited impact on answer accuracy.

Contribution

Introduces a novel Socratic-style copilot aimed at scaffolding evaluation skills, with empirical evidence on its effects and user perceptions.

Findings

Deep user engagement with the copilot

Limited improvement in answer correctness

Trade-off between chat time and exploration

Abstract

Generative AI (GenAI) tools are transforming information seeking, but their fluent, authoritative responses risk overreliance and discourage independent verification and reasoning. Rather than replacing the cognitive work of users, GenAI systems should be designed to support and scaffold it. Therefore, this paper introduces an LLM-based conversational copilot designed to scaffold information evaluation rather than provide answers and foster digital literacy skills. In a pre-registered, randomised controlled trial (N=261) examining three interface conditions including a chat-based copilot, our mixed-methods analysis reveals that users engaged deeply with the copilot, demonstrating metacognitive reflection. However, the copilot did not significantly improve answer correctness or search engagement, largely due to a "time-on-chat vs. exploration" trade-off and users' bias toward positive information. Qualitative findings reveal tension between the copilot's Socratic approach and users' desire for efficiency. These results highlight both the promise and pitfalls of pedagogical copilots, and we outline design pathways to reconcile literacy goals with efficiency demands.