Noise Perturbation for Saliency Prediction with Psychophysical Synthetic Images

TL;DR

This study compares CNN and classic models' saliency prediction performance on synthetic images with noise, revealing Fourier and spectral models outperform CNNs but are less stable, offering insights into human and artificial vision systems.

Contribution

It introduces a novel evaluation of saliency models on psychophysical synthetic images under noise, highlighting the strengths of Fourier and spectral models over CNNs.

Findings

Fourier and spectral models outperform CNNs on noisy synthetic images.

Psychophysical models are less stable under noise than CNNs.

Investigating CNNs with psychophysical methods benefits neuroscience and AI.

Abstract

Convolutional neural networks (CNNs) have achieved great success in natural image saliency prediction. The primary goal of this study is to investigate the performance of saliency prediction in CNN and classic models with psychophysical synthetic images under noise perturbation. Is it still as decent as natural images in terms of performance? In the meantime, it can be used to investigate the relationship between CNNs and human vision, mainly low-level vision functions. On the other hand, are CNNs exact replicas of human visual function? This study used CNNs, Fourier, and spectral models inspired by low-level vision systems to investigate saliency prediction on psychophysical synthetic images rather than natural images. According to our findings, saliency prediction models inspired by Fourier and spectral theory outperformed current pre-trained deep neural networks on psychophysical images with noise perturbation. However, psychophysical models were more unstable in noise than pre-trained deep neural networks. Meanwhile, we suggested that investigating CNNs with psychophysical methods could benefit visual neuroscience and artificial neural network studies.