FishFormer: Annulus Slicing-based Transformer for Fisheye Rectification with Efficacy Domain Exploration

TL;DR

FishFormer introduces an annulus slicing-based Transformer architecture tailored for fisheye image rectification, effectively capturing global and local distortion characteristics and outperforming existing methods.

Contribution

The paper proposes a novel annulus slicing method and a layer attention mechanism to improve fisheye rectification by better modeling distortion distribution and local textures.

Findings

Outperforms state-of-the-art fisheye rectification methods

Effectively models uneven distortion distribution in patches

Enhances local texture perception with layer attention

Abstract

Numerous significant progress on fisheye image rectification has been achieved through CNN. Nevertheless, constrained by a fixed receptive field, the global distribution and the local symmetry of the distortion have not been fully exploited. To leverage these two characteristics, we introduced Fishformer that processes the fisheye image as a sequence to enhance global and local perception. We tuned the Transformer according to the structural properties of fisheye images. First, the uneven distortion distribution in patches generated by the existing square slicing method confuses the network, resulting in difficult training. Therefore, we propose an annulus slicing method to maintain the consistency of the distortion in each patch, thus perceiving the distortion distribution well. Second, we analyze that different distortion parameters have their own efficacy domains. Hence, the perception of the local area is as important as the global, but Transformer has a weakness for local texture perception. Therefore, we propose a novel layer attention mechanism to enhance the local perception and texture transfer. Our network simultaneously implements global perception and focused local perception decided by the different parameters. Extensive experiments demonstrate that our method provides superior performance compared with state-of-the-art methods.