Investigating the Effects of LLM Use on Critical Thinking Under Time Constraints: Access Timing and Time Availability

TL;DR

This study explores how the timing of LLM access and available time influence critical thinking performance, revealing that temporal factors can either enhance or impair decision-making depending on the context.

Contribution

It introduces a novel experimental framework examining the interaction between LLM access timing, time constraints, and critical thinking performance.

Findings

Early LLM access improves performance under time pressure.

Late LLM access impairs performance when ample time is available.

Time constraints significantly influence LLM's impact on critical thinking.

Abstract

The impact of large language models (LLMs) on critical thinking has provoked growing attention, yet this impact on actual performance may not be uniformly negative or positive. Particularly, the role of time -- the temporal context under which an LLM is provided -- remains overlooked. In a between-subjects experiment (n=393), we examined two types of time constraints for a critical thinking task requiring participants to make a reasoned decision for a real-world scenario based on diverse documents: (1) LLM access timing -- an LLM available only at the beginning (early), throughout (continuous), near the end (late), or not at all (no LLM), and (2) time availability -- insufficient or sufficient time for the task. We found a temporal reversal: LLM access from the start (early, continuous) improved performance under time pressure but impaired it with sufficient time, whereas beginning the task independently (late, no LLM) showed the opposite pattern. These findings demonstrate that time constraints fundamentally shape whether an LLM augments or undermines critical thinking, making time a central consideration when designing LLM support and evaluating human-AI collaboration in cognitive tasks.