Interpolated SelectionConv for Spherical Images and Surfaces

TL;DR

This paper introduces Interpolated SelectionConv, a versatile framework for applying CNNs to spherical images and surfaces by representing them as graphs, enabling efficient processing without relying on specific sampling strategies.

Contribution

It proposes a novel interpolated SelectionConv method that allows existing 2D CNNs to operate on spherical and arbitrary surface data, broadening application scope.

Findings

Effective on style transfer and segmentation tasks for spherical images.

Compatible with various surface types, including complex topologies.

Fast and efficient due to leveraging existing graph implementations.

Abstract

We present a new and general framework for convolutional neural network operations on spherical (or omnidirectional) images. Our approach represents the surface as a graph of connected points that doesn't rely on a particular sampling strategy. Additionally, by using an interpolated version of SelectionConv, we can operate on the sphere while using existing 2D CNNs and their weights. Since our method leverages existing graph implementations, it is also fast and can be fine-tuned efficiently. Our method is also general enough to be applied to any surface type, even those that are topologically non-simple. We demonstrate the effectiveness of our technique on the tasks of style transfer and segmentation for spheres as well as stylization for 3D meshes. We provide a thorough ablation study of the performance of various spherical sampling strategies.