PsyPhy: A Psychophysics Driven Evaluation Framework for Visual Recognition
Brandon RichardWebster, Samuel E. Anthony, and Walter J. Scheirer

TL;DR
This paper introduces PsyPhy, a novel evaluation framework for visual recognition models based on psychophysics, which assesses perceptual thresholds through detailed stimulus-response analysis, challenging claims of human-like performance.
Contribution
PsyPhy applies psychophysical methods to evaluate visual recognition models, providing a more detailed and accurate assessment of their perceptual capabilities.
Findings
Questions recent claims of human-like performance in CNNs
Identifies specific perceptual thresholds where models fail
Offers a pathway to improve recognition algorithms
Abstract
By providing substantial amounts of data and standardized evaluation protocols, datasets in computer vision have helped fuel advances across all areas of visual recognition. But even in light of breakthrough results on recent benchmarks, it is still fair to ask if our recognition algorithms are doing as well as we think they are. The vision sciences at large make use of a very different evaluation regime known as Visual Psychophysics to study visual perception. Psychophysics is the quantitative examination of the relationships between controlled stimuli and the behavioral responses they elicit in experimental test subjects. Instead of using summary statistics to gauge performance, psychophysics directs us to construct item-response curves made up of individual stimulus responses to find perceptual thresholds, thus allowing one to identify the exact point at which a subject can no longer…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
