Pixel-variant Local Homography for Fisheye Stereo Rectification Minimizing Resampling Distortion

TL;DR

This paper introduces a pixel-wise local homography method for fisheye stereo rectification that significantly reduces resampling distortion while maintaining epipolar constraints, facilitating easier 3D reconstruction.

Contribution

It proposes a novel pixel-wise local homography approach with an optimization framework, addressing the severe resampling distortion in existing fisheye stereo rectification methods.

Findings

Reduces resampling distortion compared to existing methods

Maintains epipolar line constraints effectively

Enables easier dense stereo matching and 3D reconstruction

Abstract

Large field-of-view fisheye lens cameras have attracted more and more researchers' attention in the field of robotics. However, there does not exist a convenient off-the-shelf stereo rectification approach which can be applied directly to fisheye stereo rig. One obvious drawback of existing methods is that the resampling distortion (which is defined as the loss of pixels due to under-sampling and the creation of new pixels due to over-sampling during rectification process) is severe if we want to obtain a rectification with epipolar line (not epipolar circle) constraint. To overcome this weakness, we propose a novel pixel-wise local homography technique for stereo rectification. First, we prove that there indeed exist enough degrees of freedom to apply pixel-wise local homography for stereo rectification. Then we present a method to exploit these freedoms and the solution via an optimization framework. Finally, the robustness and effectiveness of the proposed method have been verified on real fisheye lens images. The rectification results show that the proposed approach can effectively reduce the resampling distortion in comparison with existing methods while satisfying the epipolar line constraint. By employing the proposed method, dense stereo matching and 3D reconstruction for fisheye lens camera become as easy as perspective lens cameras.