Learning Depth from Focus in the Wild

TL;DR

This paper introduces a novel CNN-based method for depth estimation from single focal stacks, featuring end-to-end inference, sharp region detection, and a realistic camera simulator, achieving top performance on benchmark datasets.

Contribution

The work presents a new depth from focus method with three key innovations: end-to-end inference with alignment, sharp region detection, and a realistic camera simulator for better generalization.

Findings

Achieves top rank in DDFF 12-Scene benchmark.

Effective on real-world images from various cameras.

Outperforms state-of-the-art methods in quantitative evaluations.

Abstract

For better photography, most recent commercial cameras including smartphones have either adopted large-aperture lens to collect more light or used a burst mode to take multiple images within short times. These interesting features lead us to examine depth from focus/defocus. In this work, we present a convolutional neural network-based depth estimation from single focal stacks. Our method differs from relevant state-of-the-art works with three unique features. First, our method allows depth maps to be inferred in an end-to-end manner even with image alignment. Second, we propose a sharp region detection module to reduce blur ambiguities in subtle focus changes and weakly texture-less regions. Third, we design an effective downsampling module to ease flows of focal information in feature extractions. In addition, for the generalization of the proposed network, we develop a simulator to realistically reproduce the features of commercial cameras, such as changes in field of view, focal length and principal points. By effectively incorporating these three unique features, our network achieves the top rank in the DDFF 12-Scene benchmark on most metrics. We also demonstrate the effectiveness of the proposed method on various quantitative evaluations and real-world images taken from various off-the-shelf cameras compared with state-of-the-art methods. Our source code is publicly available at https://github.com/wcy199705/DfFintheWild.