Generalized Closed-form Formulae for Feature-based Subpixel Alignment in Patch-based Matching

TL;DR

This paper derives closed-form formulas for subpixel disparity estimation in patch-based matching, improving accuracy and efficiency in stereo matching and optical flow by generalizing to multi-dimensional search spaces.

Contribution

It introduces novel closed-form solutions for subpixel disparity calculation for standard cost functions, extending from 1D to multi-dimensional search spaces in patch matching.

Findings

Small improvement over state-of-the-art in 1D search spaces

Significant improvement in 2D search spaces

Enhanced accuracy without expensive search procedures

Abstract

Cost-based image patch matching is at the core of various techniques in computer vision, photogrammetry and remote sensing. When the subpixel disparity between the reference patch in the source and target images is required, either the cost function or the target image have to be interpolated. While cost-based interpolation is the easiest to implement, multiple works have shown that image based interpolation can increase the accuracy of the subpixel matching, but usually at the cost of expensive search procedures. This, however, is problematic, especially for very computation intensive applications such as stereo matching or optical flow computation. In this paper, we show that closed form formulae for subpixel disparity computation for the case of one dimensional matching, e.g., in the case of rectified stereo images where the search space is of one dimension, exists when using the standard NCC, SSD and SAD cost functions. We then demonstrate how to generalize the proposed formulae to the case of high dimensional search spaces, which is required for unrectified stereo matching and optical flow extraction. We also compare our results with traditional cost volume interpolation formulae as well as with state-of-the-art cost-based refinement methods, and show that the proposed formulae bring a small improvement over the state-of-the-art cost-based methods in the case of one dimensional search spaces, and a significant improvement when the search space is two dimensional.