3D Shape Perception Integrates Intuitive Physics and Analysis-by-Synthesis

TL;DR

This paper introduces a generative analysis-by-synthesis framework for 3D shape perception that incorporates intuitive physics, outperforming neural network models in matching human perception accuracy and response times.

Contribution

It presents a novel physics-based inference model for 3D shape perception that explains both typical and atypical cases, demonstrating superior alignment with human performance.

Findings

The model best matches human accuracy and response times.

It significantly correlates with human performance on difficult discriminations.

Deep neural networks are less adequate for human-like shape perception.

Abstract

Many surface cues support three-dimensional shape perception, but people can sometimes still see shape when these features are missing -- in extreme cases, even when an object is completely occluded, as when covered with a draped cloth. We propose a framework for 3D shape perception that explains perception in both typical and atypical cases as analysis-by-synthesis, or inference in a generative model of image formation: the model integrates intuitive physics to explain how shape can be inferred from deformations it causes to other objects, as in cloth-draping. Behavioral and computational studies comparing this account with several alternatives show that it best matches human observers in both accuracy and response times, and is the only model that correlates significantly with human performance on difficult discriminations. Our results suggest that bottom-up deep neural network models are not fully adequate accounts of human shape perception, and point to how machine vision systems might achieve more human-like robustness.