PersonalityScanner: Exploring the Validity of Personality Assessment Based on Multimodal Signals in Virtual Reality

TL;DR

This paper introduces PersonalityScanner, a VR-based system that collects multimodal data during simulated daily tasks to objectively assess personality traits, addressing limitations of traditional self-report methods.

Contribution

The study develops a novel VR simulator that captures diverse multimodal signals to improve the accuracy and objectivity of personality assessment.

Findings

PersonalityScanner effectively captures multimodal signals in VR.

Multimodal data improves personality trait prediction accuracy.

The system demonstrates superior performance over traditional methods.

Abstract

Human cognition significantly influences expressed behavior and is intrinsically tied to authentic personality traits. Personality assessment plays a pivotal role in various fields, including psychology, education, social media, etc. However, traditional self-report questionnaires can only provide data based on what individuals are willing and able to disclose, thereby lacking objective. Moreover, automated measurements and peer assessments demand significant human effort and resources. In this paper, given the advantages of the Virtual Reality (VR) technique, we develop a VR simulator -- PersonalityScanner, to stimulate cognitive processes and simulate daily behaviors based on an immersive and interactive simulation environment, in which participants carry out a battery of engaging tasks that formulate a natural story of first-day at work. Through this simulator, we collect a synchronous multi-modal dataset with ten modalities, including first/third-person video, audio, text, eye tracking, facial microexpression, pose, depth data, log, and inertial measurement unit. By systematically examining the contributions of different modalities on revealing personality, we demonstrate the superior performance and effectiveness of PersonalityScanner.