DynGhost: Temporally-Modelled Transformer for Dynamic Ghost Imaging with Quantum Detectors

TL;DR

DynGhost introduces a transformer-based approach for dynamic ghost imaging that leverages temporal coherence and quantum-aware training to improve reconstruction under realistic hardware noise conditions.

Contribution

The paper proposes DynGhost, a novel transformer architecture with quantum-aware training for dynamic ghost imaging, addressing temporal coherence and realistic noise modeling.

Findings

DynGhost outperforms traditional and existing deep learning methods.

Significant improvements in dynamic and photon-starved imaging scenarios.

Effective handling of realistic hardware noise models.

Abstract

Ghost imaging reconstructs spatial information from a single-pixel bucket detector by correlating structured illumination patterns with scalar intensity measurements. While deep learning approaches have achieved promising results on static scenes, two critical limitations remain unaddressed: existing architectures fail to exploit temporal coherence across frames, leaving dynamic ghost imaging largely unsolved, and they assume additive Gaussian noise models that do not reflect the true Poissonian statistics of real single-photon hardware. We present DynGhost (Dynamic Ghost Imaging Transformer), a transformer architecture that addresses both limitations through alternating spatial and temporal attention blocks. Our quantum-aware training framework, based on physically accurate detector simulations (SNSPDs, SPADs, SiPMs) and Anscombe variance-stabilizing normalization, resolves the distribution shift that causes classical models to fail under realistic hardware constraints. Experiments across multiple benchmarks demonstrate that DynGhost outperforms both traditional reconstruction methods and existing deep learning architectures, with particular gains in dynamic and photon-starved settings.