Accurate Simulation Pipeline for Passive Single-Photon Imaging

TL;DR

This paper introduces a detailed simulation pipeline for SPAD sensors, enabling the creation of synthetic datasets like SPAD-MNIST to advance low-light imaging algorithms and facilitate training without extensive real data.

Contribution

The authors develop and validate a comprehensive SPAD simulation pipeline and generate synthetic datasets, aiding research in low-light imaging and machine learning applications.

Findings

CNN classifiers perform well on reconstructed fluxes at 5 mlux

Simulated data enables training classifiers that generalize to real SPAD images

Synthetic datasets support development of SPAD-specific processing algorithms

Abstract

Single-Photon Avalanche Diodes (SPADs) are new and promising imaging sensors. These sensors are sensitive enough to detect individual photons hitting each pixel, with extreme temporal resolution and without readout noise. Thus, SPADs stand out as an optimal choice for low-light imaging. Due to the high price and limited availability of SPAD sensors, the demand for an accurate data simulation pipeline is substantial. Indeed, the scarcity of SPAD datasets hinders the development of SPAD-specific processing algorithms and impedes the training of learning-based solutions. In this paper, we present a comprehensive SPAD simulation pipeline and validate it with multiple experiments using two recent commercial SPAD sensors. Our simulator is used to generate the SPAD-MNIST, a single-photon version of the seminal MNIST dataset, to investigate the effectiveness of convolutional neural network (CNN) classifiers on reconstructed fluxes, even at extremely low light conditions, e.g., 5 mlux. We also assess the performance of classifiers exclusively trained on simulated data on real images acquired from SPAD sensors at different light conditions. The synthetic dataset encompasses different SPAD imaging modalities and is made available for download. Project page: https://boracchi.faculty.polimi.it/Projects/SPAD-MNIST.html.