Deep Depth from Focal Stack with Defocus Model for Camera-Setting Invariance

TL;DR

This paper introduces a learning-based depth estimation method from focal stacks that remains accurate despite variations in camera settings, improving robustness and applicability in real-world scenarios.

Contribution

It presents a novel defocus model integrated with a plane sweep volume to achieve camera-setting invariance in depth from focus estimation.

Findings

Achieves state-of-the-art performance on benchmark datasets.

Robust against synthetic-to-real domain gaps.

Maintains accuracy despite different camera settings during training and testing.

Abstract

We propose a learning-based depth from focus/defocus (DFF), which takes a focal stack as input for estimating scene depth. Defocus blur is a useful cue for depth estimation. However, the size of the blur depends on not only scene depth but also camera settings such as focus distance, focal length, and f-number. Current learning-based methods without any defocus models cannot estimate a correct depth map if camera settings are different at training and test times. Our method takes a plane sweep volume as input for the constraint between scene depth, defocus images, and camera settings, and this intermediate representation enables depth estimation with different camera settings at training and test times. This camera-setting invariance can enhance the applicability of learning-based DFF methods. The experimental results also indicate that our method is robust against a synthetic-to-real domain gap, and exhibits state-of-the-art performance.