Towards Minimal Focal Stack in Shape from Focus

TL;DR

This paper introduces a physics-based augmentation and a deep network to enable shape from focus depth estimation using only two images, reducing the need for large focal stacks while maintaining accuracy.

Contribution

It proposes a novel focal stack augmentation with auxiliary cues and a deep focus volume network that allows depth estimation from just two images, improving practicality.

Findings

Augmentation improves existing SFF models' accuracy with minimal stacks.

The approach achieves state-of-the-art performance with only two images.

Experiments on synthetic and real data validate the method's effectiveness.

Abstract

Shape from Focus (SFF) is a depth reconstruction technique that estimates scene structure from focus variations observed across a focal stack, that is, a sequence of images captured at different focus settings. A key limitation of SFF methods is their reliance on densely sampled, large focal stacks, which limits their practical applicability. In this study, we propose a focal stack augmentation that enables SFF methods to estimate depth using a reduced stack of just two images, without sacrificing precision. We introduce a simple yet effective physics-based focal stack augmentation that enriches the stack with two auxiliary cues: an all-in-focus (AiF) image estimated from two input images, and Energy-of-Difference (EOD) maps, computed as the energy of differences between the AiF and input images. Furthermore, we propose a deep network that computes a deep focus volume from the augmented focal stacks and iteratively refines depth using convolutional Gated Recurrent Units (ConvGRUs) at multiple scales. Extensive experiments on both synthetic and real-world datasets demonstrate that the proposed augmentation benefits existing state-of-the-art SFF models, enabling them to achieve comparable accuracy. The results also show that our approach maintains state-of-the-art performance with a minimal stack size.