BrainCodec: Neural fMRI codec for the decoding of cognitive brain states
Yuto Nishimura, Masataka Sawayama, Ayumu Yamashita, Hideki Nakayama,, and Kaoru Amano
TL;DR
BrainCodec is a neural fMRI codec that improves mental state decoding, enhances signal-to-noise ratio, and provides interpretability for brain activity analysis, representing a significant advancement in neuroimaging data processing.
Contribution
We introduce BrainCodec, a novel neural codec for fMRI data that improves compression, decoding accuracy, and interpretability, and can serve as a denoising tool for neuroimaging.
Findings
BrainCodec outperforms previous methods in mental state decoding.
fMRI reconstructions with BrainCodec increase signal-to-noise ratio.
Latent representations reveal insights into brain state differences.
Abstract
Recently, leveraging big data in deep learning has led to significant performance improvements, as confirmed in applications like mental state decoding using fMRI data. However, fMRI datasets remain relatively small in scale, and the inherent issue of low signal-to-noise ratios (SNR) in fMRI data further exacerbates these challenges. To address this, we apply compression techniques as a preprocessing step for fMRI data. We propose BrainCodec, a novel fMRI codec inspired by the neural audio codec. We evaluated BrainCodec's compression capability in mental state decoding, demonstrating further improvements over previous methods. Furthermore, we analyzed the latent representations obtained through BrainCodec, elucidating the similarities and differences between task and resting state fMRI, highlighting the interpretability of BrainCodec. Additionally, we demonstrated that fMRI…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
- Novel application of audio compression techniques to fMRI data - Building up on one of the largest "foundation models" - Improved performance on mental state decoding tasks when combined with existing models - Comprehensive ablation studies and hyperparameter analysis - Code and trained models made publicly available
- Lack of comparison to relevant baseline methods like temporal ICA or sparse dictionary approaches - Inconsistent performance improvements across all datasets (e.g., MDTB showed decreased performance) - Potential overfitting concerns when using small models like Linear+BrainCodec - The paper's organization, clarity, or depth of discussion may not have met the high standards expected for the conference, for example "Our contributions" section as presented does not effectively communicate novel c
- The paper is generally well-organized and easy to follow. - The motivation and objectives of the study are clearly articulated and compelling. - The compression approach proposed in this study appears promising.
- While the authors compare BrainCodec with BrainVAE and the baseline methods, a more comprehensive comparison with other fMRI compression techniques, such as dictionary learning-based methods, could provide a deeper understanding of the relative strengths and weaknesses. - The authors do not fully address the limitations of the current study, unless I missed something.
The paper has some strengths, such as the innovative introduction of BrainCodec as a compression technique for fMRI data, which addresses important challenges like small dataset sizes and low signal-to-noise ratios (SNR). The approach shows promise in improving mental state decoding performance and offers insights into latent representations, highlighting differences between task and resting state fMRI. Additionally, the suggestion that BrainCodec could function as a denoising method indicates p
The paper presents several limitations that undermine its overall contribution. Firstly, it fails to clearly distinguish the contributions of BrainCodec from existing methods, particularly those discussed in Thomas et al. (2022), including Causal State Modeling (CSM) and linear approaches. This lack of differentiation raises questions about the novelty and significance of the proposed method. Secondly, the evaluation lacks a thorough comparison with established techniques in fMRI analysis, makin
1. The proposed BrainCodec is novel, integrating some prevalent techniques such as Residual Vector Quantization and Causal Sequence Modeling. 2. Based on reported empirical experiments, BrainCodeC can reconstruct fMRI signals better than BrainVAE.
The reviewers have highlighted several weaknesses in this work, which can be categorized as follows: 1). Missing Details: This concern encompasses Questions 1, 5, and 6, where critical information appears to be lacking. 2). Poor Motivation: This issue pertains to Question 2, where the rationale behind the codebook is not thoroughly explained. Additionally, reconstructing fMRI data does not provide comparable insights to those offered by fMRI analytics and diagnostics. 3). Technological Issues
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEEG and Brain-Computer Interfaces · Functional Brain Connectivity Studies