Shallow and Deep Convolutional Networks for Saliency Prediction

TL;DR

This paper introduces the first end-to-end convolutional neural networks for saliency prediction, demonstrating that data-driven deep learning approaches can outperform traditional hand-crafted methods in speed and accuracy.

Contribution

It presents two novel CNN architectures for saliency prediction, one shallow trained from scratch and one deeper with transfer learning, pioneering end-to-end deep learning in this domain.

Findings

Deep CNNs achieve high accuracy in saliency prediction

End-to-end training outperforms traditional methods

Deep models are computationally efficient

Abstract

The prediction of salient areas in images has been traditionally addressed with hand-crafted features based on neuroscience principles. This paper, however, addresses the problem with a completely data-driven approach by training a convolutional neural network (convnet). The learning process is formulated as a minimization of a loss function that measures the Euclidean distance of the predicted saliency map with the provided ground truth. The recent publication of large datasets of saliency prediction has provided enough data to train end-to-end architectures that are both fast and accurate. Two designs are proposed: a shallow convnet trained from scratch, and a another deeper solution whose first three layers are adapted from another network trained for classification. To the authors knowledge, these are the first end-to-end CNNs trained and tested for the purpose of saliency prediction.