Detecting Reading-Induced Confusion Using EEG and Eye Tracking

TL;DR

This study combines EEG and eye tracking to detect reading-induced confusion, demonstrating improved classification accuracy and highlighting neural markers, paving the way for adaptive systems in learning and HCI.

Contribution

It introduces a multimodal approach using EEG and eye tracking to identify confusion during natural reading, with machine learning models enhancing detection accuracy.

Findings

Multimodal models improve confusion detection accuracy by 4-22%.

Neural signatures of confusion are prominent in the brain's temporal regions.

Average detection accuracy reaches 77.3%, with a maximum of 89.6%.

Abstract

Humans regularly navigate an overwhelming amount of information via text media, whether reading articles, browsing social media, or interacting with chatbots. Confusion naturally arises when new information conflicts with or exceeds a reader's comprehension or prior knowledge, posing a challenge for learning. In this study, we present a multimodal investigation of reading-induced confusion using EEG and eye tracking. We collected neural and gaze data from 11 adult participants as they read short paragraphs sampled from diverse, real-world sources. By isolating the N400 event-related potential (ERP), a well-established neural marker of semantic incongruence, and integrating behavioral markers from eye tracking, we provide a detailed analysis of the neural and behavioral correlates of confusion during naturalistic reading. Using machine learning, we show that multimodal (EEG + eye tracking) models improve classification accuracy by 4-22% over unimodal baselines, reaching an average weighted participant accuracy of 77.3% and a best accuracy of 89.6%. Our results highlight the dominance of the brain's temporal regions in these neural signatures of confusion, suggesting avenues for wearable, low-electrode brain-computer interfaces (BCI) for real-time monitoring. These findings lay the foundation for developing adaptive systems that dynamically detect and respond to user confusion, with potential applications in personalized learning, human-computer interaction, and accessibility.