Level Set Binocular Stereo with Occlusions

TL;DR

This paper presents a novel level-set based stereo matching method that explicitly models occlusion geometry to improve boundary accuracy in stereo images, especially in figure-ground scenes.

Contribution

It introduces an energy and level-set optimizer that encodes occlusion geometry, enhancing boundary detection in stereo matching for two-layer scenes.

Findings

More accurate boundaries than previous occlusion-handling stereo methods.

Applicable to figure-ground scenes from Middlebury and Falling Things datasets.

Demonstrates potential for human-like occlusion cue integration in stereo systems.

Abstract

Localizing stereo boundaries and predicting nearby disparities are difficult because stereo boundaries induce occluded regions where matching cues are absent. Most modern computer vision algorithms treat occlusions secondarily (e.g., via left-right consistency checks after matching) or rely on high-level cues to improve nearby disparities (e.g., via deep networks and large training sets). They ignore the geometry of stereo occlusions, which dictates that the spatial extent of occlusion must equal the amplitude of the disparity jump that causes it. This paper introduces an energy and level-set optimizer that improves boundaries by encoding occlusion geometry. Our model applies to two-layer, figure-ground scenes, and it can be implemented cooperatively using messages that pass predominantly between parents and children in an undecimated hierarchy of multi-scale image patches. In a small collection of figure-ground scenes curated from Middlebury and Falling Things stereo datasets, our model provides more accurate boundaries than previous occlusion-handling stereo techniques. This suggests new directions for creating cooperative stereo systems that incorporate occlusion cues in a human-like manner.