Convolutions on Spherical Images

TL;DR

This paper introduces a novel spherical image representation using the ISEA projection, significantly enhancing convolutional neural network performance on spherical data, especially in semantic segmentation tasks.

Contribution

It proposes a new spherical image representation based on the ISEA projection, outperforming existing methods in convolutional neural networks for spherical images.

Findings

Improved semantic segmentation accuracy by 12.6%

Outperforms state-of-the-art methods on spherical image tasks

Provides a theoretically grounded optimal spherical image representation

Abstract

Applying convolutional neural networks to spherical images requires particular considerations. We look to the millennia of work on cartographic map projections to provide the tools to define an optimal representation of spherical images for the convolution operation. We propose a representation for deep spherical image inference based on the icosahedral Snyder equal-area (ISEA) projection, a projection onto a geodesic grid, and show that it vastly exceeds the state-of-the-art for convolution on spherical images, improving semantic segmentation results by 12.6%.