Fully Self-Supervised Depth Estimation from Defocus Clue

TL;DR

This paper introduces a fully self-supervised depth estimation method from sparse focal stacks, eliminating the need for ground-truth depth or all-in-focus images, thus enabling practical real-world applications.

Contribution

It proposes a novel self-supervised framework for depth-from-defocus that works without ground-truth data and uses an optical model for validation and refinement.

Findings

Outperforms existing methods on benchmark datasets

Works effectively with real focal stacks

Provides a strong baseline for self-supervised DFD

Abstract

Depth-from-defocus (DFD), modeling the relationship between depth and defocus pattern in images, has demonstrated promising performance in depth estimation. Recently, several self-supervised works try to overcome the difficulties in acquiring accurate depth ground-truth. However, they depend on the all-in-focus (AIF) images, which cannot be captured in real-world scenarios. Such limitation discourages the applications of DFD methods. To tackle this issue, we propose a completely self-supervised framework that estimates depth purely from a sparse focal stack. We show that our framework circumvents the needs for the depth and AIF image ground-truth, and receives superior predictions, thus closing the gap between the theoretical success of DFD works and their applications in the real world. In particular, we propose (i) a more realistic setting for DFD tasks, where no depth or AIF image ground-truth is available; (ii) a novel self-supervision framework that provides reliable predictions of depth and AIF image under the challenging setting. The proposed framework uses a neural model to predict the depth and AIF image, and utilizes an optical model to validate and refine the prediction. We verify our framework on three benchmark datasets with rendered focal stacks and real focal stacks. Qualitative and quantitative evaluations show that our method provides a strong baseline for self-supervised DFD tasks.