Tangent Images for Mitigating Spherical Distortion

TL;DR

This paper introduces tangent images, a novel spherical image representation that reduces distortion, enabling scalable, transferable, and efficient 360° computer vision applications without specialized kernels.

Contribution

The work presents a new tangent image method inspired by cartography, allowing standard CNNs to effectively process high-resolution spherical images and transfer from perspective training.

Findings

Comparable performance to specialized spherical kernels

Enables transfer learning from perspective images

Improves feature detection quality on spherical images

Abstract

In this work, we propose "tangent images," a spherical image representation that facilitates transferable and scalable $360^\circ$ computer vision. Inspired by techniques in cartography and computer graphics, we render a spherical image to a set of distortion-mitigated, locally-planar image grids tangent to a subdivided icosahedron. By varying the resolution of these grids independently of the subdivision level, we can effectively represent high resolution spherical images while still benefiting from the low-distortion icosahedral spherical approximation. We show that training standard convolutional neural networks on tangent images compares favorably to the many specialized spherical convolutional kernels that have been developed, while also scaling efficiently to handle significantly higher spherical resolutions. Furthermore, because our approach does not require specialized kernels, we show that we can transfer networks trained on perspective images to spherical data without fine-tuning and with limited performance drop-off. Finally, we demonstrate that tangent images can be used to improve the quality of sparse feature detection on spherical images, illustrating its usefulness for traditional computer vision tasks like structure-from-motion and SLAM.