GazeMind: A Gaze-Guided LLM Agent for Personalized Cognitive Load Assessment

TL;DR

GazeMind is a novel gaze-guided LLM framework for personalized, interpretable cognitive load assessment on smart glasses, generalizing across scenarios without fine-tuning and supported by a large new dataset.

Contribution

It introduces GazeMind, a gaze-guided LLM system that generalizes cognitive load assessment across scenarios and personalizes predictions without fine-tuning.

Findings

GazeMind outperforms baselines by over 20% across all metrics.

It provides interpretable cognitive load predictions.

The CogLoad-Bench dataset includes 152 participants and over 10,000 annotations.

Abstract

Smart glasses with AI assistants are increasingly used in daily life. However, current systems lack awareness of the user's internal cognitive state, leaving them unable to proactively anticipate users' needs without access to cognitive load. Existing methods for assessing cognitive load either rely on impractical sensors for lightweight eyewear or utilize eye gaze-based models that suffer from poor interpretability, and require task-specific fine-tuning, often failing to generalize across individuals. We propose GazeMind, a gaze-guided LLM agent framework for cognitive load assessment on smart glasses. It encodes eye-tracking data into structured representations for LLM-based reasoning and provides interpretable cognitive load predictions. Importantly, GazeMind generalizes across scenarios without LLM fine-tuning through a novel task-guidance reasoning approach and achieves personalized adaptation by incorporating user-specific characteristics and historical references. To support evaluation, we introduce CogLoad-Bench, the largest gaze-based cognitive load dataset with 152 participants, 40+ hours of multimodal data, and 10K+ real-time annotations across controlled and real-world tasks. Experiments show that GazeMind achieves state-of-the-art performance, outperforming baselines by over 20% across all metrics.