Brain Functional Connectivity under Teleoperation Latency: a fNIRS Study

TL;DR

This study investigates how latency in teleoperation affects brain connectivity and performance, revealing that latency increases cognitive load and that real-time haptic feedback can mitigate neural and performance impairments.

Contribution

It provides novel neural evidence linking teleoperation latency to increased cognitive load and demonstrates the mitigating effect of real-time haptic feedback on brain connectivity.

Findings

Latency increases functional connectivity in prefrontal and motor cortices.

Real-time haptic feedback reduces neural connectivity associated with cognitive load.

Performance worsens with latency and improves with haptic feedback.

Abstract

Objective: This study aims to understand the cognitive impact of latency in teleoperation and the related mitigation methods, using functional Near-Infrared Spectroscopy (fNIRS) to analyze functional connectivity. Background: Latency between command, execution, and feedback in teleoperation can impair performance and affect operators mental state. The neural underpinnings of these effects are not well understood. Method: A human subject experiment (n = 41) of a simulated remote robot manipulation task was performed. Three conditions were tested: no latency, with visual and haptic latency, with visual latency and no haptic latency. fNIRS and performance data were recorded and analyzed. Results: The presence of latency in teleoperation significantly increased functional connectivity within and between prefrontal and motor cortexes. Maintaining visual latency while providing real-time haptic feedback reduced the average functional connectivity in all cortical networks and showed a significantly different connectivity ratio within prefrontal and motor cortical networks. The performance results showed the worst performance in the all-delayed condition and best performance in no latency condition, which echoes the neural activity patterns. Conclusion: The study provides neurological evidence that latency in teleoperation increases cognitive load, anxiety, and challenges in motion planning and control. Real-time haptic feedback, however, positively influences neural pathways related to cognition, decision-making, and sensorimotor processes. Application: This research can inform the design of ergonomic teleoperation systems that mitigate the effects of latency.