Empowering Functional Neuroimaging: A Pre-trained Generative Framework for Unified Representation of Neural Signals

TL;DR

This paper introduces a generative AI framework that creates a unified representation of multimodal neuroimaging data, reducing costs, addressing data scarcity, and improving fairness in brain-computer interface decoding.

Contribution

It presents a novel unified representation framework that generates data for constrained modalities and underrepresented groups, enhancing neuroimaging analysis and BCI fairness.

Findings

Generated data aligns with real brain activity patterns

Improves downstream BCI decoding performance

Enhances fairness by augmenting underrepresented groups

Abstract

Multimodal functional neuroimaging enables systematic analysis of brain mechanisms and provides discriminative representations for brain-computer interface (BCI) decoding. However, its acquisition is constrained by high costs and feasibility limitations. Moreover, underrepresentation of specific groups undermines fairness of BCI decoding model. To address these challenges, we propose a unified representation framework for multimodal functional neuroimaging via generative artificial intelligence (AI). By mapping multimodal functional neuroimaging into a unified representation space, the proposed framework is capable of generating data for acquisition-constrained modalities and underrepresented groups. Experiments show that the framework can generate data consistent with real brain activity patterns, provide insights into brain mechanisms, and improve performance on downstream tasks. More importantly, it can enhance model fairness by augmenting data for underrepresented groups. Overall, the framework offers a new paradigm for decreasing the cost of acquiring multimodal functional neuroimages and enhancing the fairness of BCI decoding models.