Scene-adaptive Coded Apertures Imaging

TL;DR

This paper introduces a scene-adaptive coded aperture imaging method that dynamically adjusts aperture patterns based on scene content, improving depth recovery and all-focus imaging by leveraging scene structure and temporal correlation.

Contribution

It proposes a novel adaptive coded aperture scheme that uses scene geometry and temporal propagation to enhance depth and focus imaging performance.

Findings

Improved depth map quality and all-focus images in experiments.

Effective scene-adaptive pattern switching based on edge directions.

Enhanced robustness in both synthetic and real scenes.

Abstract

Coded aperture imaging systems have recently shown great success in recovering scene depth and extending the depth-of-field. The ideal pattern, however, would have to serve two conflicting purposes: 1) be broadband to ensure robust deconvolution and 2) has sufficient zero-crossings for a high depth discrepancy. This paper presents a simple but effective scene-adaptive coded aperture solution to bridge this gap. We observe that the geometric structures in a natural scene often exhibit only a few edge directions, and the successive frames are closely correlated. Therefore we adopt a spatial partitioning and temporal propagation scheme. In each frame, we address one principal direction by applying depth-discriminative codes along it and broadband codes along its orthogonal direction. Since within a frame only the regions with edge direction corresponding to its aperture code behaves well, we utilize the close among-frame correlation to propagate the high quality single frame results temporally to obtain high performance over the whole image lattice. To physically implement this scheme, we use a Liquid Crystal on Silicon (LCoS) microdisplay that permits fast changing pattern codes. Firstly, we capture the scene with a pinhole and analyze the scene content to determine primary edge orientations. Secondly, we sequentially apply the proposed coding scheme with these orientations in the following frames. Experiments on both synthetic and real scenes show that our technique is able to combine advantages of the state-of-the-art patterns for recovering better quality depth map and all-focus images.