Continuous 3D Label Stereo Matching using Local Expansion Moves

TL;DR

This paper introduces a novel local expansion move-based stereo matching algorithm that efficiently infers 3D plane labels, achieving state-of-the-art results on benchmark datasets through localization and spatial propagation techniques.

Contribution

It proposes a new local expansion move scheme for graph cut-based stereo matching that improves efficiency and accuracy by combining localization and spatial propagation.

Findings

Achieves top performance on Middlebury stereo benchmarks.

Produces smooth, piecewise linear disparity maps.

Efficiently infers continuous 3D labels using randomized search.

Abstract

We present an accurate stereo matching method using local expansion moves based on graph cuts. This new move-making scheme is used to efficiently infer per-pixel 3D plane labels on a pairwise Markov random field (MRF) that effectively combines recently proposed slanted patch matching and curvature regularization terms. The local expansion moves are presented as many alpha-expansions defined for small grid regions. The local expansion moves extend traditional expansion moves by two ways: localization and spatial propagation. By localization, we use different candidate alpha-labels according to the locations of local alpha-expansions. By spatial propagation, we design our local alpha-expansions to propagate currently assigned labels for nearby regions. With this localization and spatial propagation, our method can efficiently infer MRF models with a continuous label space using randomized search. Our method has several advantages over previous approaches that are based on fusion moves or belief propagation; it produces submodular moves deriving a subproblem optimality; it helps find good, smooth, piecewise linear disparity maps; it is suitable for parallelization; it can use cost-volume filtering techniques for accelerating the matching cost computations. Even using a simple pairwise MRF, our method is shown to have best performance in the Middlebury stereo benchmark V2 and V3.