Grayscale to Hyperspectral at Any Resolution Using a Phase-Only Lens

TL;DR

This paper introduces a novel method to reconstruct high-quality hyperspectral images from single grayscale snapshots using a phase-only lens and a diffusion model, enabling efficient, high-resolution spectral imaging with uncertainty estimation.

Contribution

First model to produce high-quality hyperspectral reconstructions from single grayscale images using a phase-only lens and diffusion models, with effective patch-based inference and uncertainty quantification.

Findings

Patch sizes as small as the PSF support achieve excellent results.

Local optical cues are sufficient for full spectral information capture.

Model provides uncertainty estimates strongly correlated with reconstruction error.

Abstract

We consider the problem of reconstructing a HxWx31 hyperspectral image from a HxW grayscale snapshot measurement that is captured using only a single diffractive optic and a filterless panchromatic photosensor. This problem is severely ill-posed, but we present the first model that produces high-quality results. We make efficient use of limited data by training a conditional denoising diffusion model that operates on small patches in a shift-invariant manner. During inference, we synchronize per-patch hyperspectral predictions using guidance derived from the optical point spread function. Surprisingly, our experiments reveal that patch sizes as small as the PSFs support achieve excellent results, and they show that local optical cues are sufficient to capture full spectral information. Moreover, by drawing multiple samples, our model provides per-pixel uncertainty estimates that strongly correlate with reconstruction error. Our work lays the foundation for a new class of high-resolution snapshot hyperspectral imagers that are compact and light-efficient.