Confidence Without Competence in AI-Assisted Knowledge Work

TL;DR

This study explores how different AI-assisted interaction designs influence students' reflection, confidence, and learning outcomes when using large language models in knowledge work.

Contribution

The paper introduces Deep3, a novel web-based system with three interaction modes designed to promote deeper thinking without increasing cognitive load.

Findings

Standard LLM interaction led to high perceived understanding but low actual learning.

Future-self explanations increased cognitive effort but improved accuracy of self-assessment.

Guided hints enhanced learning gains with minimal frustration.

Abstract

Large Language Models (LLMs) are widely used by students, yet their tendency to provide fast and complete answers may discourage reflection and foster overconfidence. We examined how alternative LLM interaction designs support deeper thinking without excessively increasing cognitive burden. We conducted a two-phase mixed-methods study. In Phase 1, interviews with 16 Gen Z students informed the design of Deep3, a web-based system with three interaction modes: \emph{a)} future-self explanations, \emph{b)} contrastive learning, and \emph{c)} guided hints. In Phase 2, we evaluated Deep3 with 85 participants across two learning tasks. We found that a standard single-agent baseline produced high perceived understanding despite the lowest objective learning. In contrast, future-self explanations imposed higher cognitive workload yet yielded the closest alignment between perceived and actual understanding, while guided hints achieved the largest learning gains without a proportional increase in frustration. These findings show that effort, confidence, and learning systematically diverge in LLM-supported work.