Fast non parametric entropy estimation for spatial-temporal saliency method

TL;DR

This paper introduces a fast nonparametric method for estimating spatial-temporal saliency based on center-surround conditional entropy, enabling practical computation of saliency maps in high-dimensional spaces.

Contribution

It presents a novel efficient technique using k-d partitioning for nonparametric entropy estimation, applicable to both still images and videos for visual saliency detection.

Findings

Competitive with state-of-the-art saliency methods on eye tracking datasets

Effective in computing spatiotemporal saliency for videos

Demonstrates practical real-time applicability

Abstract

This paper formulates bottom-up visual saliency as center surround conditional entropy and presents a fast and efficient technique for the computation of such a saliency map. It is shown that the new saliency formulation is consistent with self-information based saliency, decision-theoretic saliency and Bayesian definition of surprises but also faces the same significant computational challenge of estimating probability density in very high dimensional spaces with limited samples. We have developed a fast and efficient nonparametric method to make the practical implementation of these types of saliency maps possible. By aligning pixels from the center and surround regions and treating their location coordinates as random variables, we use a k-d partitioning method to efficiently estimating the center surround conditional entropy. We present experimental results on two publicly available eye tracking still image databases and show that the new technique is competitive with state of the art bottom-up saliency computational methods. We have also extended the technique to compute spatiotemporal visual saliency of video and evaluate the bottom-up spatiotemporal saliency against eye tracking data on a video taken onboard a moving vehicle with the driver's eye being tracked by a head mounted eye-tracker.