FisheyeHDK: Hyperbolic Deformable Kernel Learning for Ultra-Wide Field-of-View Image Recognition

TL;DR

This paper introduces FisheyeHDK, a novel hyperbolic deformable kernel learning method that improves CNN performance on ultra-wide field-of-view fisheye images by better modeling distortions.

Contribution

It proposes a hybrid hyperbolic and Euclidean CNN architecture for deformable kernels, addressing limitations of Euclidean geometry in fisheye image recognition.

Findings

Outperforms existing deformable kernel methods on fisheye images.

Effective on synthetic distortion profiles and real fisheye camera data.

Improves accuracy in ultra-wide FoV image recognition.

Abstract

Conventional convolution neural networks (CNNs) trained on narrow Field-of-View (FoV) images are the state-of-the-art approaches for object recognition tasks. Some methods proposed the adaptation of CNNs to ultra-wide FoV images by learning deformable kernels. However, they are limited by the Euclidean geometry and their accuracy degrades under strong distortions caused by fisheye projections. In this work, we demonstrate that learning the shape of convolution kernels in non-Euclidean spaces is better than existing deformable kernel methods. In particular, we propose a new approach that learns deformable kernel parameters (positions) in hyperbolic space. FisheyeHDK is a hybrid CNN architecture combining hyperbolic and Euclidean convolution layers for positions and features learning. First, we provide an intuition of hyperbolic space for wide FoV images. Using synthetic distortion profiles, we demonstrate the effectiveness of our approach. We select two datasets - Cityscapes and BDD100K 2020 - of perspective images which we transform to fisheye equivalents at different scaling factors (analog to focal lengths). Finally, we provide an experiment on data collected by a real fisheye camera. Validations and experiments show that our approach improves existing deformable kernel methods for CNN adaptation on fisheye images.