Single-pixel imaging via data-driven and deep image prior dual networks

TL;DR

This paper introduces a dual-network iterative optimization framework for single-pixel imaging that combines deep image prior and data-driven networks, achieving high-quality reconstructions with fewer iterations and enhanced detail preservation.

Contribution

It proposes a novel dual-network approach that integrates DIP-Net and DD-Net, along with a residual block for better information transfer, improving efficiency and generalization in SPI.

Findings

Fewer iteration steps needed for high-quality reconstruction.

Enhanced image details at low sampling rates.

Effective in both indoor and outdoor experiments.

Abstract

Single-pixel imaging(SPI),especially when integrated with deep neural networks like deep image prior networks (DIP-Net) or data-driven networks (DD-Net), has gained considerable attention for its capability to generate high-quality reconstructed images, even in the presence of sub-sampling conditions. However, DIP-Net often requires thousands of iterations to achieve high-quality image reconstruction, and DD-Net performs optimally only when the target closely resembles the features present in its training set. To overcome these limitations, we propose a dual-network iterative optimization (SPI-DNIO) framework that combines the strengths of both DD-Net and DIP-Net. It has been demonstrated that this approach can recover high-quality images with fewer iteration steps. Furthermore, to address the challenge of SPI inputs having less effective information at low sampling rates, we have designed a residual block enriched with gradient information, which can convey details to deeper layers, thereby enhancing the deep network's learning capabilities. We have applied these techniques to both indoor experiments with active lighting and outdoor long-range experiments with passive lighting. Our experimental results confirm the exceptional reconstruction capabilities and generalization performance of the SPI-DNIO framework.