Time-Resolved MNIST Dataset for Single-Photon Recognition

TL;DR

This paper introduces a realistic simulation process for time-resolved SPAD imaging, generating a new dataset based on MNIST to facilitate research in low-light, time-resolved photon detection and classification.

Contribution

It presents a novel simulation framework for time-resolved SPAD imaging and creates a new dataset for advancing research in photon-based image recognition.

Findings

Generated a realistic time-resolved MNIST dataset

Enabled research in low-light photon detection

Facilitated CNN classifier evaluation in photon-limited scenarios

Abstract

Time-resolved single photon imaging is a promising imaging modality characterized by the unique capability of timestamping the arrivals of single photons. Single-Photon Avalanche Diodes (SPADs) are the leading technology for implementing modern time-resolved pixels, suitable for passive imaging with asynchronous readout. However, they are currently limited to small sized arrays, thus there is a lack of datasets for passive time-resolved SPAD imaging, which in turn hinders research on this peculiar imaging data. In this paper we describe a realistic simulation process for SPAD imaging, which takes into account both the stochastic nature of photon arrivals and all the noise sources involved in the acquisition process of time-resolved SPAD arrays. We have implemented this simulator in a software prototype able to generate arbitrary-sized time-resolved SPAD arrays operating in passive mode. Starting from a reference image, our simulator generates a realistic stream of timestamped photon detections. We use our simulator to generate a time-resolved version of MNIST, which we make publicly available. Our dataset has the purpose of encouraging novel research directions in time-resolved SPAD imaging, as well as investigating the performance of CNN classifiers in extremely low-light conditions.