Cognitive Assessment and Training in Extended Reality: Multimodal Systems, Clinical Utility, and Current Challenges

TL;DR

This paper reviews how extended reality technologies like VR, AR, and MR are revolutionizing cognitive assessment and training by providing immersive, multimodal environments, while discussing current challenges and future opportunities for clinical and research applications.

Contribution

It offers a comprehensive overview of XR-based cognitive assessment and training, highlighting advantages, current limitations, and future directions for integrating multimodal data and improving usability.

Findings

XR enhances ecological validity and engagement in cognitive tasks.

Multimodal data collection enables deeper understanding of cognitive processes.

Challenges include usability, cybersickness, and accessibility barriers.

Abstract

Extended reality (XR) technologies-encompassing virtual reality (VR), augmented reality (AR), and mixed reality (MR) are transforming cognitive assessment and training by offering immersive, interactive environments that simulate real-world tasks. XR enhances ecological validity while enabling real-time, multimodal data collection through tools such as galvanic skin response (GSR), electroencephalography (EEG), eye tracking (ET), hand tracking, and body tracking. This allows for a more comprehensive understanding of cognitive and emotional processes, as well as adaptive, personalized interventions for users. Despite these advancements, current XR applications often underutilize the full potential of multimodal integration, relying primarily on visual and auditory inputs. Challenges such as cybersickness, usability concerns, and accessibility barriers further limit the widespread adoption of XR tools in cognitive science and clinical practice. This review examines XR-based cognitive assessment and training, focusing on its advantages over traditional methods, including ecological validity, engagement, and adaptability. It also explores unresolved challenges such as system usability, cost, and the need for multimodal feedback integration. The review concludes by identifying opportunities for optimizing XR tools to improve cognitive evaluation and rehabilitation outcomes, particularly for diverse populations, including older adults and individuals with cognitive impairments.