SelectionConv: Convolutional Neural Networks for Non-rectilinear Image Data

TL;DR

This paper introduces SelectionConv, a graph convolution operator that enables the use of pre-trained CNN weights on non-rectilinear image data by converting such data into graphs, thus broadening CNN applicability.

Contribution

It presents a novel graph convolution method that transfers traditional CNN weights to irregular data domains without extensive retraining.

Findings

Effective transfer of pre-trained CNNs to non-rectilinear data

Successful applications in segmentation, stylization, and depth prediction

Operates on various irregular image domains without large datasets

Abstract

Convolutional Neural Networks have revolutionized vision applications. There are image domains and representations, however, that cannot be handled by standard CNNs (e.g., spherical images, superpixels). Such data are usually processed using networks and algorithms specialized for each type. In this work, we show that it may not always be necessary to use specialized neural networks to operate on such spaces. Instead, we introduce a new structured graph convolution operator that can copy 2D convolution weights, transferring the capabilities of already trained traditional CNNs to our new graph network. This network can then operate on any data that can be represented as a positional graph. By converting non-rectilinear data to a graph, we can apply these convolutions on these irregular image domains without requiring training on large domain-specific datasets. Results of transferring pre-trained image networks for segmentation, stylization, and depth prediction are demonstrated for a variety of such data forms.