Rediscovering Affordance: A Reinforcement Learning Perspective

TL;DR

This paper introduces a reinforcement learning-based theory of affordance-formation, explaining how users discover and adapt to object actions through experience, and demonstrates it with a virtual robot model.

Contribution

It presents a novel integrative theory linking affordance-formation to reinforcement learning, bridging cognitive science and human-computer interaction.

Findings

The virtual robot model adapts affordances in interactive tasks.

Predictions align with human data but humans adapt faster.

The theory explains how affordances are learned and generalized.

Abstract

Affordance refers to the perception of possible actions allowed by an object. Despite its relevance to human-computer interaction, no existing theory explains the mechanisms that underpin affordance-formation; that is, how affordances are discovered and adapted via interaction. We propose an integrative theory of affordance-formation based on the theory of reinforcement learning in cognitive sciences. The key assumption is that users learn to associate promising motor actions to percepts via experience when reinforcement signals (success/failure) are present. They also learn to categorize actions (e.g., "rotating" a dial), giving them the ability to name and reason about affordance. Upon encountering novel widgets, their ability to generalize these actions determines their ability to perceive affordances. We implement this theory in a virtual robot model, which demonstrates human-like adaptation of affordance in interactive widgets tasks. While its predictions align with trends in human data, humans are able to adapt affordances faster, suggesting the existence of additional mechanisms.