Upright adjustment with graph convolutional networks

TL;DR

This paper introduces a new method combining CNN and GCN for upright adjustment of 360 images, utilizing a spherical representation and a novel loss function, outperforming previous fully connected approaches.

Contribution

The paper presents a novel CNN-GCN architecture with a spherical graph representation and a new loss function for improved 360 image upright adjustment.

Findings

Outperforms fully connected based methods

Effective spherical representation of images

Novel loss function improves accuracy

Abstract

We present a novel method for the upright adjustment of 360 images. Our network consists of two modules, which are a convolutional neural network (CNN) and a graph convolutional network (GCN). The input 360 images is processed with the CNN for visual feature extraction, and the extracted feature map is converted into a graph that finds a spherical representation of the input. We also introduce a novel loss function to address the issue of discrete probability distributions defined on the surface of a sphere. Experimental results demonstrate that our method outperforms fully connected based methods.