Prediction-based attention computing: a proof of concept study

TL;DR

This study introduces a new adaptive XR system that adjusts simulations based on users' brain predictions, showing it can influence sensorimotor responses and surprise reactions.

Contribution

The paper introduces prediction-based attention computing (PbAC), a novel adaptive XR framework that modulates simulations based on users' internal state predictions.

Findings

Sensorimotor responses were influenced by the expectedness of XR stimuli in PbAC conditions.

PbAC elicited surprisal responses comparable to or greater than control conditions when high prediction errors were induced.

Abstract

Recent advancements in extended reality (XR) and data modelling present new opportunities for adaptive simulation solutions, which can measure and respond to individual neuropsychological states. However, questions remain about the optimal metrics for real-time data capture and the applicability of these solutions for enhancing user experiences. The present research examined a novel form of adaptive XR, called “prediction-based attention computing” (PbAC), which tailors simulations based on computational models of the brain and, thus, the dynamic sensorimotor processes theorised to underpin human perception and learning. Specifically, this study aimed to demonstrate whether PbAC can adaptively capture users’ internal state predictions and modulate associated neuropsychological responses. To test this, we used an XR-based racquetball paradigm, in which participants were tasked with intercepting virtual balls that emerged from different starting locations. For PbAC conditions, in-situ eye tracking data assessments were utilised to index participant’s prior beliefs and manipulate levels of expectedness (i.e., prediction error) on each trial. Various measures of predictive sensorimotor behaviour were then extracted and compared with data from probability-controlled and matched-order control conditions. Results showed that sensorimotor responses were affected by the expectedness of XR stimuli, and that clear, prediction-related biases emerged within PbAC conditions. The novel computing software also provoked marked surprisal responses on trials designed to elicit high levels of prediction error, and these surprisal effects were similar, or even greater than, those in our comparison conditions. Together, the findings provide proof of concept for PbAC and support its development within future research and technology innovations.

The online version contains supplementary material available at 10.1007/s10055-025-01307-w.

Figures