DCBAN: A Dynamic Confidence Bayesian Adaptive Network for Reconstructing Visual Images from fMRI Signals

TL;DR

This paper introduces DCBAN, a new neural network that improves the accuracy and naturalness of reconstructing visual images from brain scans.

Contribution

The novel Dynamic Confidence Bayesian Adaptive Network (DCBAN) enhances image reconstruction from fMRI data using adaptive regularization and confidence-based diffusion.

Findings

DCBAN outperforms existing methods in structural and semantic visual image reconstruction from fMRI signals.

The model achieves state-of-the-art performance on the NSD dataset with improved PixCorr, Incep, and CLIP scores.

Dynamic confidence and Bayesian adaptation modules significantly enhance image detail and naturalness.

Abstract

Background: Current fMRI (functional magnetic resonance imaging)-driven brain information decoding for visual image reconstruction techniques faces issues such as poor structural fidelity, inadequate model generalization, and unnatural visual image reconstruction in complex scenarios. Methods: To address these challenges, this study proposes a Dynamic Confidence Bayesian Adaptive Network (DCBAN). In this network model, deep nested Singular Value Decomposition is introduced to embed low-rank constraints into the deep learning model layers for fine-grained feature extraction, thus improving structural fidelity. The proposed Bayesian Adaptive Fractional Ridge Regression module, based on singular value space, dynamically adjusts the regularization parameters, significantly enhancing the decoder’s generalization ability under complex stimulus conditions. The constructed Dynamic Confidence Adaptive Diffusion Model module incorporates a confidence network and time decay strategy, dynamically adjusting the semantic injection strength during the generation phase, further enhancing the details and naturalness of the generated images. Results: The proposed DCBAN method is applied to the NSD, outperforming state-of-the-art methods by 8.41%, 0.6%, and 4.8% in PixCorr (0.361), Incep (96.0%), and CLIP (97.8%), respectively, achieving the current best performance in both structural and semantic fMRI visual image reconstruction. Conclusions: The DCBAN proposed in this thesis offers a novel solution for reconstructing visual images from fMRI signals, significantly enhancing the robustness and generative quality of the reconstructed images.

Figures