Affine Correspondences in Stereo Vision: Theory, Practice, and Limitations

TL;DR

This paper explores the use of affine correspondences in stereo vision, providing theoretical insights, practical estimation techniques, and evaluating their impact on 3D reconstruction accuracy using both synthetic and real data.

Contribution

It introduces novel methods for estimating local affine transformations from image directions and analyzes their effect on surface normal reconstruction accuracy.

Findings

Estimation accuracy is around a few degrees in realistic scenarios.

Affine correspondences improve 3D surface normal estimation.

Evaluation includes synthetic and real-world stereo configurations.

Abstract

Affine transformations have been recently used for stereo vision. They can be exploited in various computer vision application, e.g., when estimating surface normals, homographies, fundamental and essential matrices. Even full 3D reconstruction can be obtained by using affine correspondences. First, this paper overviews the fundamental statements for affine transformations and epipolar geometry. Then it is investigated how the transformation accuracy influences the quality of the 3D reconstruction. Besides, we propose novel techniques for estimating the local affine transformation from corresponding image directions; moreover, the fundamental matrix, related to the processed image pair, can also be exploited. Both synthetic and real quantitative evaluations are implemented based on the accuracy of the reconstructed surface normals. For the latter one, a special object, containing three perpendicular planes with chessboard patterns, is constructed. The quantitative evaluations are based on the accuracy of the reconstructed surface normals and it is concluded that the estimation accuracy is around a few degrees for realistic test cases. Special stereo poses and plane orientations are also evaluated in detail.