Image Matching Filtering and Refinement by Planes and Beyond

TL;DR

This paper presents a modular, geometry-based filtering and refinement method for sparse image correspondences that does not rely on deep learning, using local homographies and plane assumptions to improve matching accuracy and robustness.

Contribution

The paper introduces a novel non-deep learning approach combining local homography-based clustering and patch refinement, with strategies to handle violations of planar assumptions, enhancing image matching robustness.

Findings

Effective in handling outliers and incompatible matches.

Refinement improves keypoint accuracy for corner-like features.

Performs well even without camera intrinsics in real-world scenarios.

Abstract

This paper introduces a modular, non-deep learning method for filtering and refining sparse correspondences in image matching. Assuming that motion flow within the scene can be approximated by local homography transformations, matches are aggregated into overlapping clusters corresponding to virtual planes using an iterative RANSAC-based approach discarding incompatible correspondences. Moreover, the underlying planar structural design provides an explicit map between local patches associated with the matches, by which optionally refine the keypoint positions through cross-correlation template matching after the patch reprojection. Finally, to enhance robustness and fault-tolerance against violations of the piece-wise planar approximation assumption, a further strategy is designed in order to minimize the relative patch distortion in the plane reprojection by introducing an intermediate homography that projects both patches into a common plane. The proposed method is extensively evaluated on standard datasets and image matching pipelines, and compared with state-of-the-art approaches. Unlike other current comparisons, the proposed benchmark also takes into account the more general, real, and practical cases where camera intrinsics are unavailable. Experimental results demonstrate that our proposed non-deep learning, geometry-based filter is effective in presence of outliers and the optional cross-correlation refinement step is valid in the case of corner-like keypoints. Finally, this study suggests that there is still significant development potential in practical image matching solutions in the considered research direction, which could be in the future incorporated in novel deep image matching architectures.