Characterizing Personality from Eye-Tracking: The Role of Gaze and Its Absence in Interactive Search Environments

TL;DR

This study demonstrates that personality traits can be effectively predicted from eye-tracking data during search tasks, especially when including gaze absence periods, using a multimodal time-series model with minimal data preprocessing.

Contribution

It introduces a novel approach that incorporates gaze missingness into multimodal time-series models to predict personality traits from search behavior.

Findings

High predictive accuracy for all Big Five traits.

Gaze missingness improves model performance.

Including both gaze signals and missingness yields 10-15% higher accuracy.

Abstract

Personality traits influence how individuals engage, behave, and make decisions during the information-seeking process. However, few studies have linked personality to observable search behaviors. This study aims to characterize personality traits through a multimodal time-series model that integrates eye-tracking data and gaze missingness-periods when the user's gaze is not captured. This approach is based on the idea that people often look away when they think, signaling disengagement or reflection. We conducted a user study with 25 participants, who used an interactive application on an iPad, allowing them to engage with digital artifacts from a museum. We rely on raw gaze data from an eye tracker, minimizing preprocessing so that behavioral patterns can be preserved without substantial data cleaning. From this perspective, we trained models to predict personality traits using gaze signals. Our results from a five-fold cross-validation study demonstrate strong predictive performance across all five dimensions: Neuroticism (Macro F1 = 77.69%), Conscientiousness (74.52%), Openness (77.52%), Agreeableness (73.09%), and Extraversion (76.69%). The ablation study examines whether the absence of gaze information affects the model performance, demonstrating that incorporating missingness improves multimodal time-series modeling. The full model, which integrates both time-series signals and missingness information, achieves 10-15% higher accuracy and macro F1 scores across all Big Five traits compared to the model without time-series signals and missingness. These findings provide evidence that personality can be inferred from search-related gaze behavior and demonstrate the value of incorporating missing gaze data into time-series multimodal modeling.