SaltiNet: Scan-path Prediction on 360 Degree Images using Saliency Volumes

TL;DR

SaltiNet is a deep neural network that predicts scanpaths on 360-degree images by utilizing a novel saliency volume representation, demonstrating advantages in scanpath prediction and related tasks.

Contribution

Introduces SaltiNet, a neural network with a saliency volume representation for improved scanpath prediction on 360-degree images.

Findings

SaltiNet effectively predicts scanpaths using saliency volumes.

Saliency volumes enhance performance over traditional methods.

The approach is applicable to related visual tasks.

Abstract

We introduce SaltiNet, a deep neural network for scanpath prediction trained on 360-degree images. The model is based on a temporal-aware novel representation of saliency information named the saliency volume. The first part of the network consists of a model trained to generate saliency volumes, whose parameters are fit by back-propagation computed from a binary cross entropy (BCE) loss over downsampled versions of the saliency volumes. Sampling strategies over these volumes are used to generate scanpaths over the 360-degree images. Our experiments show the advantages of using saliency volumes, and how they can be used for related tasks. Our source code and trained models available at https://github.com/massens/saliency-360salient-2017.