Faster gaze prediction with dense networks and Fisher pruning

TL;DR

This paper introduces Fisher pruning, a method to significantly reduce the size of deep networks for gaze prediction, achieving 10x faster inference without sacrificing accuracy, thus enabling real-time applications.

Contribution

The paper presents Fisher pruning combined with knowledge distillation to create more efficient saliency prediction models, a novel approach for reducing overparameterization in this domain.

Findings

Achieved 10x speedup in gaze prediction inference.

Maintained state-of-the-art AUC performance on CAT2000.

Demonstrated importance for real-world and video saliency applications.

Abstract

Predicting human fixations from images has recently seen large improvements by leveraging deep representations which were pretrained for object recognition. However, as we show in this paper, these networks are highly overparameterized for the task of fixation prediction. We first present a simple yet principled greedy pruning method which we call Fisher pruning. Through a combination of knowledge distillation and Fisher pruning, we obtain much more runtime-efficient architectures for saliency prediction, achieving a 10x speedup for the same AUC performance as a state of the art network on the CAT2000 dataset. Speeding up single-image gaze prediction is important for many real-world applications, but it is also a crucial step in the development of video saliency models, where the amount of data to be processed is substantially larger.