EEG-fused Digital Twin Brain for Autonomous Driving in Virtual Scenarios

TL;DR

This paper introduces a Bayesian framework that fuses EEG and MRI data to create a digital twin brain model, enabling real-time autonomous driving simulation and advancing understanding of sensorimotor integration.

Contribution

It presents a novel method for integrating high-temporal EEG with high-spatial MRI data to build a biologically realistic digital twin brain for autonomous driving.

Findings

Achieved biologically plausible EEG signal generation with high correlation to real data.

Demonstrated successful autonomous driving in a simulated environment using decoded steering signals.

Outperformed chance and empirical signals in steering prediction accuracy.

Abstract

Current methodologies typically integrate biophysical brain models with functional magnetic resonance imaging(fMRI) data - while offering millimeter-scale spatial resolution (0.5-2 mm^3 voxels), these approaches suffer from limited temporal resolution (>0.5 Hz) for tracking rapid neural dynamics during continuous tasks. Conversely, Electroencephalogram (EEG) provides millisecond-scale temporal precision (<=1 ms sampling rate) for real-time guidance of continuous task execution, albeit constrained by low spatial resolution. To reconcile these complementary modalities, we present a generalizable Bayesian inference framework that integrates high-spatial-resolution structural MRI(sMRI) with high-temporal-resolution EEG to construct a biologically realistic digital twin brain(DTB) model. The framework establishes voxel-wise mappings between millisecond-scale EEG and sMRI-derived spiking networks, while demonstrating its translational potential through a brain-inspired autonomous driving simulation. Our EEG-DTB model achieves capabilities: (1) Biologically-plausible EEG signal generation (0.88 resting-state,0.60 task-state correlation), with simulated signals in task-state yielding steering predictions outperforming both chance and empirical signals (p<0.05); (2) Successful autonomous driving in the CARLA simulator using decoded steering angles. The proposed approach pioneers a new paradigm for studying sensorimotor integration and for mechanistic studies of perception-action cycles and the development of brain-inspired control systems.