Joint Image and Depth Estimation with Mask-Based Lensless Cameras

TL;DR

This paper introduces a novel joint image and depth estimation method for mask-based lensless cameras, capable of handling scenes with large depth variations, demonstrated through simulations and real-world experiments.

Contribution

It proposes an alternating gradient descent algorithm for continuous depth and light estimation, improving robustness over existing methods for scenes with diverse depths.

Findings

The algorithm outperforms existing methods in natural scenes with large depth ranges.

Simulation results show accurate 3D scene reconstruction.

Experimental results validate the approach on real objects.

Abstract

Mask-based lensless cameras replace the lens of a conventional camera with a custom mask. These cameras can potentially be very thin and even flexible. Recently, it has been demonstrated that such mask-based cameras can recover light intensity and depth information of a scene. Existing depth recovery algorithms either assume that the scene consists of a small number of depth planes or solve a sparse recovery problem over a large 3D volume. Both these approaches fail to recover the scenes with large depth variations. In this paper, we propose a new approach for depth estimation based on an alternating gradient descent algorithm that jointly estimates a continuous depth map and light distribution of the unknown scene from its lensless measurements. We present simulation results on image and depth reconstruction for a variety of 3D test scenes. A comparison between the proposed algorithm and other method shows that our algorithm is more robust for natural scenes with a large range of depths. We built a prototype lensless camera and present experimental results for reconstruction of intensity and depth maps of different real objects.