Efficient Surface-Aware Semi-Global Matching with Multi-View Plane-Sweep Sampling

TL;DR

This paper introduces an efficient hierarchical algorithm that enhances semi-global matching by incorporating local surface orientations, enabling accurate depth and normal map estimation in oblique aerial imagery with real-time processing capabilities.

Contribution

The work extends semi-global matching with a multi-view plane-sweep approach and local surface orientation modeling, improving accuracy for slanted surfaces in oblique aerial images.

Findings

Comparable accuracy to offline SfM pipelines like COLMAP

Achieves online, incremental processing at 1-2Hz

Effectively models slanted surfaces in aerial imagery

Abstract

Online augmentation of an oblique aerial image sequence with structural information is an essential aspect in the process of 3D scene interpretation and analysis. One key aspect in this is the efficient dense image matching and depth estimation. Here, the Semi-Global Matching (SGM) approach has proven to be one of the most widely used algorithms for efficient depth estimation, providing a good trade-off between accuracy and computational complexity. However, SGM only models a first-order smoothness assumption, thus favoring fronto-parallel surfaces. In this work, we present a hierarchical algorithm that allows for efficient depth and normal map estimation together with confidence measures for each estimate. Our algorithm relies on a plane-sweep multi-image matching followed by an extended SGM optimization that allows to incorporate local surface orientations, thus achieving more consistent and accurate estimates in areasmade up of slanted surfaces, inherent to oblique aerial imagery. We evaluate numerous configurations of our algorithm on two different datasets using an absolute and relative accuracy measure. In our evaluation, we show that the results of our approach are comparable to the ones achieved by refined Structure-from-Motion (SfM) pipelines, such as COLMAP, which are designed for offline processing. In contrast, however, our approach only considers a confined image bundle of an input sequence, thus allowing to perform an online and incremental computation at 1Hz-2Hz.