gQIR: Generative Quanta Image Reconstruction

TL;DR

This paper introduces gQIR, a novel method that adapts large diffusion models to reconstruct high-quality images from sparse, noisy photon detection data in low-light conditions, outperforming existing approaches.

Contribution

The paper presents a new approach that integrates latent diffusion models with photon statistics handling for quanta burst imaging, enabling high-quality reconstructions in photon-limited scenarios.

Findings

Significant perceptual quality improvements over classical methods.

Effective handling of Bernoulli photon noise in image reconstruction.

Successful application to real-world color SPAD datasets and challenging video benchmarks.

Abstract

Capturing high-quality images from only a few detected photons is a fundamental challenge in computational imaging. Single-photon avalanche diode (SPAD) sensors promise high-quality imaging in regimes where conventional cameras fail, but raw \emph{quanta frames} contain only sparse, noisy, binary photon detections. Recovering a coherent image from a burst of such frames requires handling alignment, denoising, and demosaicing (for color) under noise statistics far outside those assumed by standard restoration pipelines or modern generative models. We present an approach that adapts large text-to-image latent diffusion models to the photon-limited domain of quanta burst imaging. Our method leverages the structural and semantic priors of internet-scale diffusion models while introducing mechanisms to handle Bernoulli photon statistics. By integrating latent-space restoration with burst-level spatio-temporal reasoning, our approach produces reconstructions that are both photometrically faithful and perceptually pleasing, even under high-speed motion. We evaluate the method on synthetic benchmarks and new real-world datasets, including the first color SPAD burst dataset and a challenging \textit{Deforming (XD)} video benchmark. Across all settings, the approach substantially improves perceptual quality over classical and modern learning-based baselines, demonstrating the promise of adapting large generative priors to extreme photon-limited sensing. Code at \href{https://github.com/Aryan-Garg/gQIR}{https://github.com/Aryan-Garg/gQIR}.