Mind-to-Face: Neural-Driven Photorealistic Avatar Synthesis via EEG Decoding

TL;DR

Mind-to-Face introduces a novel neural framework that decodes EEG signals into photorealistic, dynamic facial avatars, enabling emotion-aware telepresence and overcoming occlusion limitations in expressive avatar systems.

Contribution

This work is the first to directly translate EEG signals into high-fidelity facial expressions for avatar synthesis, integrating neural decoding with advanced rendering techniques.

Findings

EEG signals can reliably predict detailed facial expressions.

The model captures subtle emotional dynamics and fine-scale geometry.

Photorealistic, view-consistent facial avatars are achievable from neural data.

Abstract

Current expressive avatar systems rely heavily on visual cues, failing when faces are occluded or when emotions remain internal. We present Mind-to-Face, the first framework that decodes non-invasive electroencephalogram (EEG) signals directly into high-fidelity facial expressions. We build a dual-modality recording setup to obtain synchronized EEG and multi-view facial video during emotion-eliciting stimuli, enabling precise supervision for neural-to-visual learning. Our model uses a CNN-Transformer encoder to map EEG signals into dense 3D position maps, capable of sampling over 65k vertices, capturing fine-scale geometry and subtle emotional dynamics, and renders them through a modified 3D Gaussian Splatting pipeline for photorealistic, view-consistent results. Through extensive evaluation, we show that EEG alone can reliably predict dynamic, subject-specific facial expressions, including subtle emotional responses, demonstrating that neural signals contain far richer affective and geometric information than previously assumed. Mind-to-Face establishes a new paradigm for neural-driven avatars, enabling personalized, emotion-aware telepresence and cognitive interaction in immersive environments.