Affordances Provide a Fundamental Categorization Principle for Visual Scenes

TL;DR

This study demonstrates that the actions a visual scene affords are a fundamental principle for categorizing scenes, surpassing object-based or visual feature-based models in explaining human perception.

Contribution

It introduces the idea that affordances are the primary basis for scene categorization, supported by large-scale experimental evidence.

Findings

Affordance similarity explains most of the structure in scene categorization.

Affordances outperform object and visual feature models in predicting perception.

Nearly half of the variance in scene categorization is explained solely by affordances.

Abstract

How do we know that a kitchen is a kitchen by looking? Relatively little is known about how we conceptualize and categorize different visual environments. Traditional models of visual perception posit that scene categorization is achieved through the recognition of a scene's objects, yet these models cannot account for the mounting evidence that human observers are relatively insensitive to the local details in an image. Psychologists have long theorized that the affordances, or actionable possibilities of a stimulus are pivotal to its perception. To what extent are scene categories created from similar affordances? Using a large-scale experiment using hundreds of scene categories, we show that the activities afforded by a visual scene provide a fundamental categorization principle. Affordance-based similarity explained the majority of the structure in the human scene categorization patterns, outperforming alternative similarities based on objects or visual features. We all models were combined, affordances provided the majority of the predictive power in the combined model, and nearly half of the total explained variance is captured only by affordances. These results challenge many existing models of high-level visual perception, and provide immediately testable hypotheses for the functional organization of the human perceptual system.