Natural Image Reconstruction from fMRI using Deep Learning: A Survey

TL;DR

This survey reviews recent deep learning approaches for reconstructing natural images from fMRI brain data, analyzing architectures, datasets, metrics, and discussing future research directions.

Contribution

It provides a comprehensive overview and performance comparison of current methods, highlighting strengths, limitations, and future challenges in fMRI-based image reconstruction.

Findings

Deep learning methods vary in architecture and performance.

Benchmark datasets and metrics are standardized for fair comparison.

Current methods have limitations in reconstruction accuracy and generalization.

Abstract

With the advent of brain imaging techniques and machine learning tools, much effort has been devoted to building computational models to capture the encoding of visual information in the human brain. One of the most challenging brain decoding tasks is the accurate reconstruction of the perceived natural images from brain activities measured by functional magnetic resonance imaging (fMRI). In this work, we survey the most recent deep learning methods for natural image reconstruction from fMRI. We examine these methods in terms of architectural design, benchmark datasets, and evaluation metrics and present a fair performance evaluation across standardized evaluation metrics. Finally, we discuss the strengths and limitations of existing studies and present potential future directions.