Human perception in computer vision

TL;DR

This paper investigates the parallels between human visual perception and deep neural network computations, finding correlations at different processing stages and suggesting architecture-independent principles of visual learning.

Contribution

It demonstrates that human perceptual sensitivities correspond to specific DNN computation stages, supporting the idea of shared principles in visual learning across biological and artificial systems.

Findings

Perceptual sensitivity to image changes correlates with DNN mid-computation.

Sensitivity to segmentation, crowding, and shape correlates with DNN end-computation.

Results support using DNNs to estimate perceptual loss and suggest architecture-independent visual learning principles.

Abstract

Computer vision has made remarkable progress in recent years. Deep neural network (DNN) models optimized to identify objects in images exhibit unprecedented task-trained accuracy and, remarkably, some generalization ability: new visual problems can now be solved more easily based on previous learning. Biological vision (learned in life and through evolution) is also accurate and general-purpose. Is it possible that these different learning regimes converge to similar problem-dependent optimal computations? We therefore asked whether the human system-level computation of visual perception has DNN correlates and considered several anecdotal test cases. We found that perceptual sensitivity to image changes has DNN mid-computation correlates, while sensitivity to segmentation, crowding and shape has DNN end-computation correlates. Our results quantify the applicability of using DNN computation to estimate perceptual loss, and are consistent with the fascinating theoretical view that properties of human perception are a consequence of architecture-independent visual learning.