Compressed-domain visual saliency models: A comparative study

TL;DR

This study compares eleven compressed-domain visual saliency models and two pixel-domain models, demonstrating that accurate saliency prediction can be achieved using only partially decoded video data, with insights into effective strategies and future challenges.

Contribution

It provides a comprehensive comparison of compressed-domain saliency models against pixel-domain models, highlighting effective strategies and identifying challenges for future research.

Findings

Compressed-domain models can achieve high accuracy using partial video data.

Certain strategies are more successful in compressed-domain saliency modeling.

The study identifies challenges and potential improvements in compressed-domain saliency estimation.

Abstract

Computational modeling of visual saliency has become an important research problem in recent years, with applications in video quality estimation, video compression, object tracking, retargeting, summarization, and so on. While most visual saliency models for dynamic scenes operate on raw video, several models have been developed for use with compressed-domain information such as motion vectors and transform coefficients. This paper presents a comparative study of eleven such models as well as two high-performing pixel-domain saliency models on two eye-tracking datasets using several comparison metrics. The results indicate that highly accurate saliency estimation is possible based only on a partially decoded video bitstream. The strategies that have shown success in compressed-domain saliency modeling are highlighted, and certain challenges are identified as potential avenues for further improvement.