QueryCDR: Query-Based Controllable Distortion Rectification Network for Fisheye Images

TL;DR

QueryCDR introduces a novel query-based approach with learnable controls to improve fisheye image rectification across varying distortion levels without retraining, enhancing practical applicability.

Contribution

The paper proposes a new query-based network with learnable control queries and modulating blocks to generalize fisheye distortion rectification across different degrees.

Findings

Achieves high-quality rectification on datasets with various distortion levels.

Demonstrates superior generalization compared to existing models.

Provides controllable rectification process for better user guidance.

Abstract

Fisheye image rectification aims to correct distortions in images taken with fisheye cameras. Although current models show promising results on images with a similar degree of distortion as the training data, they will produce sub-optimal results when the degree of distortion changes and without retraining. The lack of generalization ability for dealing with varying degrees of distortion limits their practical application. In this paper, we take one step further to enable effective distortion rectification for images with varying degrees of distortion without retraining. We propose a novel Query-Based Controllable Distortion Rectification network for fisheye images (QueryCDR). In particular, we first present the Distortion-aware Learnable Query Mechanism (DLQM), which defines the latent spatial relationships for different distortion degrees as a series of learnable queries. Each query can be learned to obtain position-dependent rectification control conditions, providing control over the rectification process. Then, we propose two kinds of controllable modulating blocks to enable the control conditions to guide the modulation of the distortion features better. These core components cooperate with each other to effectively boost the generalization ability of the model at varying degrees of distortion. Extensive experiments on fisheye image datasets with different distortion degrees demonstrate our approach achieves high-quality and controllable distortion rectification.