ZEBRA: Towards Zero-Shot Cross-Subject Generalization for Universal Brain Visual Decoding

TL;DR

ZEBRA introduces a zero-shot brain visual decoding framework that disentangles subject-related and semantic representations, enabling accurate cross-subject reconstruction without subject-specific training.

Contribution

It is the first framework to achieve zero-shot generalization in brain visual decoding by explicitly disentangling subject and semantic components using adversarial training.

Findings

Outperforms zero-shot baselines significantly

Achieves comparable performance to finetuned models

Demonstrates effective subject-invariant semantic decoding

Abstract

Recent advances in neural decoding have enabled the reconstruction of visual experiences from brain activity, positioning fMRI-to-image reconstruction as a promising bridge between neuroscience and computer vision. However, current methods predominantly rely on subject-specific models or require subject-specific fine-tuning, limiting their scalability and real-world applicability. In this work, we introduce ZEBRA, the first zero-shot brain visual decoding framework that eliminates the need for subject-specific adaptation. ZEBRA is built on the key insight that fMRI representations can be decomposed into subject-related and semantic-related components. By leveraging adversarial training, our method explicitly disentangles these components to isolate subject-invariant, semantic-specific representations. This disentanglement allows ZEBRA to generalize to unseen subjects without any additional fMRI data or retraining. Extensive experiments show that ZEBRA significantly outperforms zero-shot baselines and achieves performance comparable to fully finetuned models on several metrics. Our work represents a scalable and practical step toward universal neural decoding. Code and model weights are available at: https://github.com/xmed-lab/ZEBRA.