Neural and Cognitive Impacts of AI: The Influence of Task Subjectivity on Human-LLM Collaboration

TL;DR

This study investigates how AI assistants like Copilot influence users' mental states and performance across different task types, revealing that effectiveness varies with task subjectivity and cognitive engagement.

Contribution

It provides a comprehensive analysis of the cognitive and physiological effects of AI assistants on humans, highlighting task-dependent variations in collaboration outcomes.

Findings

AI reduces workload and increases enjoyment in objective tasks.

No physiological changes observed due to AI assistance.

AI's usefulness decreases in tasks involving episodic memory.

Abstract

AI-based interactive assistants are advancing human-augmenting technology, yet their effects on users' mental and physiological states remain under-explored. We address this gap by analyzing how Copilot for Microsoft Word, a LLM-based assistant, impacts users. Using tasks ranging from objective (SAT reading comprehension) to subjective (personal reflection), and with measurements including fNIRS, Empatica E4, NASA-TLX, and questionnaires, we measure Copilot's effects on users. We also evaluate users' performance with and without Copilot across tasks. In objective tasks, participants reported a reduction of workload and an increase in enjoyment, which was paired with objective performance increases. Participants reported reduced workload and increased enjoyment with no change in performance in a creative poetry writing task. However, no benefits due to Copilot use were reported in a highly subjective self-reflection task. Although no physiological changes were recorded due to Copilot use, task-dependent differences in prefrontal cortex activation offer complementary insights into the cognitive processes associated with successful and unsuccessful human-AI collaboration. These findings suggest that AI assistants' effectiveness varies with task type-particularly showing decreased usefulness in tasks that engage episodic memory-and presents a brain-network based hypothesis of human-AI collaboration.