Motion estimation for fisheye video sequences combining perspective projection with camera calibration information

TL;DR

This paper introduces a novel motion estimation method for fisheye videos that combines perspective projection with camera calibration, improving accuracy and luminance PSNR in real-world applications.

Contribution

It presents a calibration and re-mapping approach that enhances existing hybrid motion estimation techniques for fisheye videos, addressing boundary distortions.

Findings

Up to 0.58 dB PSNR gain with calibration and re-mapping.

Overall up to 3.32 dB PSNR improvement over traditional methods.

Effective handling of ultra-wide fisheye angles.

Abstract

Fisheye cameras prove a convenient means in surveillance and automotive applications as they provide a very wide field of view for capturing their surroundings. Contrary to typical rectilinear imagery, however, fisheye video sequences follow a different mapping from the world coordinates to the image plane which is not considered in standard video processing techniques. In this paper, we present a motion estimation method for real-world fisheye videos by combining perspective projection with knowledge about the underlying fisheye projection. The latter is obtained by camera calibration since actual lenses rarely follow exact models. Furthermore, we introduce a re-mapping for ultra-wide angles which would otherwise lead to wrong motion compensation results for the fisheye boundary. Both concepts extend an existing hybrid motion estimation method for equisolid fisheye video sequences that decides between traditional and fisheye block matching in a block-based manner. Compared to that method, the proposed calibration and re-mapping extensions yield gains of up to 0.58 dB in luminance PSNR for real-world fisheye video sequences. Overall gains amount to up to 3.32 dB compared to traditional block matching.