Progress Towards Decoding Visual Imagery via fNIRS

TL;DR

This paper explores reconstructing visual images from fNIRS brain activity, demonstrating that cm-scale spatial resolution is sufficient for image retrieval, and presents a prototype design for a time-domain fNIRS device.

Contribution

It shows that image reconstruction is feasible with low-resolution fNIRS data and provides a prototype design for a time-domain fNIRS system for brain imaging.

Findings

71% retrieval accuracy with 1-cm resolution

Time-domain fNIRS can achieve 1-cm resolution

Prototype design for a time-domain fNIRS device

Abstract

We demonstrate the possibility of reconstructing images from fNIRS brain activity and start building a prototype to match the required specs. By training an image reconstruction model on downsampled fMRI data, we discovered that cm-scale spatial resolution is sufficient for image generation. We obtained 71% retrieval accuracy with 1-cm resolution, compared to 93% on the full-resolution fMRI, and 20% with 2-cm resolution. With simulations and high-density tomography, we found that time-domain fNIRS can achieve 1-cm resolution, compared to 2-cm resolution for continuous-wave fNIRS. Lastly, we share designs for a prototype time-domain fNIRS device, consisting of a laser driver, a single photon detector, and a time-to-digital converter system.