Guided Lensless Polarization Imaging

TL;DR

This paper presents a novel RGB-guided lensless polarization imaging system that enhances reconstruction quality by combining a compact polarization sensor with a conventional RGB camera and a two-stage deep learning pipeline.

Contribution

It introduces a two-stage reconstruction method using physics-based inversion and Transformer-based fusion, improving lensless polarization imaging quality with RGB guidance.

Findings

Significantly improves reconstruction quality over lensless-only methods.

Generalizes well across different datasets and conditions.

Achieves high-quality real-world results without fine-tuning.

Abstract

Polarization imaging captures the polarization state of light, revealing information invisible to the human eye yet valuable in domains such as biomedical diagnostics, autonomous driving, and remote sensing. However, conventional polarization cameras are often expensive, bulky, or both, limiting their practical use. Lensless imaging offers a compact, low-cost alternative by replacing the lens with a simple optical element like a diffuser and performing computational reconstruction, but existing lensless polarization systems suffer from limited reconstruction quality. To overcome these limitations, we introduce a RGB-guided lensless polarization imaging system that combines a compact polarization-RGB sensor with an auxiliary, widely available conventional RGB camera providing structural guidance. We reconstruct multi-angle polarization images for each RGB color channel through a two-stage pipeline: a physics-based inversion recovers an initial polarization image, followed by a Transformer-based fusion network that refines this reconstruction using the RGB guidance image from the conventional RGB camera. Our two-stage method significantly improves reconstruction quality and fidelity over lensless-only baselines, generalizes across datasets and imaging conditions, and achieves high-quality real-world results on our physical prototype lensless camera without any fine-tuning.