RoFIR: Robust Fisheye Image Rectification Framework Impervious to Optical Center Deviation

TL;DR

This paper introduces RoFIR, a novel framework for fisheye image rectification that effectively handles optical center deviations by learning local distortion patterns, outperforming existing methods.

Contribution

The paper proposes a distortion vector map (DVM) and a two-stage training paradigm to extend fisheye rectification to deviated images, addressing global distortion variability.

Findings

Outperforms existing rectification methods on deviated fisheye images.

Effective in both central and deviated fisheye image correction.

Data augmentation improves model robustness across various fisheye types.

Abstract

Fisheye images are categorized fisheye into central and deviated based on the optical center position. Existing rectification methods are limited to central fisheye images, while this paper proposes a novel method that extends to deviated fisheye image rectification. The challenge lies in the variant global distortion distribution pattern caused by the random optical center position. To address this challenge, we propose a distortion vector map (DVM) that measures the degree and direction of local distortion. By learning the DVM, the model can independently identify local distortions at each pixel without relying on global distortion patterns. The model adopts a pre-training and fine-tuning training paradigm. In the pre-training stage, it predicts the distortion vector map and perceives the local distortion features of each pixel. In the fine-tuning stage, it predicts a pixel-wise flow map for deviated fisheye image rectification. We also propose a data augmentation method mixing central, deviated, and distorted-free images. Such data augmentation promotes the model performance in rectifying both central and deviated fisheye images, compared with models trained on single-type fisheye images. Extensive experiments demonstrate the effectiveness and superiority of the proposed method.