ManifoldFormer: Geometric Deep Learning for Neural Dynamics on Riemannian Manifolds

TL;DR

ManifoldFormer introduces a geometric deep learning framework that models neural signals on Riemannian manifolds, significantly improving EEG analysis accuracy and generalization by respecting the brain's intrinsic geometric structure.

Contribution

This work presents a novel geometric deep learning architecture combining Riemannian VAE, geodesic-aware Transformer, and neural ODEs for neural manifold learning and dynamics modeling.

Findings

4.6-4.8% higher accuracy over state-of-the-art methods

6.2-10.2% higher Cohen's Kappa

Enhanced cross-subject generalization

Abstract

Existing EEG foundation models mainly treat neural signals as generic time series in Euclidean space, ignoring the intrinsic geometric structure of neural dynamics that constrains brain activity to low-dimensional manifolds. This fundamental mismatch between model assumptions and neural geometry limits representation quality and cross-subject generalization. ManifoldFormer addresses this limitation through a novel geometric deep learning framework that explicitly learns neural manifold representations. The architecture integrates three key innovations: a Riemannian VAE for manifold embedding that preserves geometric structure, a geometric Transformer with geodesic-aware attention mechanisms operating directly on neural manifolds, and a dynamics predictor leveraging neural ODEs for manifold-constrained temporal evolution. Extensive evaluation across four public datasets demonstrates substantial improvements over state-of-the-art methods, with 4.6-4.8% higher accuracy and 6.2-10.2% higher Cohen's Kappa, while maintaining robust cross-subject generalization. The geometric approach reveals meaningful neural patterns consistent with neurophysiological principles, establishing geometric constraints as essential for effective EEG foundation models.