A Turing Test for Artificial Nets devoted to model Human Vision

TL;DR

This paper proposes a rigorous testing framework for ANN models of human vision, using a new low-level dataset based on visual neuroscience facts to evaluate model behavior against human visual responses.

Contribution

It introduces a low-level dataset of retina-V1 pathway facts and evaluates three recent models, highlighting their similarities and differences in mimicking human visual behavior.

Findings

Model 1 closely matches human receptive fields.

Model 1 better captures nonlinear spatio-chromatic behaviors.

The dataset reveals differences in model behaviors for low-level visual facts.

Abstract

In our invited talk at the AI Evaluation Workshop of the University of Bristol back in June 2022 we argued that, despite claims about successful modeling of the visual brain using ANNs, the problem is far from being solved (even for low-level vision). Open issues include: where should we read from ANNs to reproduce human behavior?, this ad-hoc read-out is part of the brain model or not?, should we use artificial psychophysics or artificial physiology?, artificial experiments should literally match the experiments in humans?. There is a clear need of rigorous procedures for experimental tests for ANNs models of the visual brain, and more generally, to understand ANNs devoted to generic vision tasks. Following our experience in using low-level facts from Visual Neuroscience in Image Processing, we presented the idea of developing a low-level dataset compiling the basic spatio-temporal and chromatic facts that are known to happen in the retina-V1 pathway, and they are not currently available in existing databases such as BrainScore. In our results we checked the behavior of three recently proposed models with similar architecture: (1) A parametric model tuned via Maximum Differentiation [Malo & Simoncelli SPIE 15, Martinez et al. PLOS 18, Martinez et al. Front. Neurosci. 19], (2) A non-parametric model, the PerceptNet, tuned to maximize the correlation with human opinion on subjective distortions [Hepburn et al. IEEE ICIP 20], and (3) A model with the same encoder as PerceptNet, but tuned for segmentation (published later as Hernandez-Camara et al. Patt.Recogn.Lett. 23, Hernandez-Camara et al. Neurocomp. 25). Results on 10 compelling psycho/physio visual facts show that the first model is the one with closer behavior to the humans in terms of receptive fields, but more interestingly, on the nonlinear behavior for spatio-chromatic patterns of a range of luminances and contrasts.