Detecting Affordances by Visuomotor Simulation

TL;DR

This paper presents a cognitive architecture that detects object affordances through visuomotor simulation, enabling a robot to distinguish passable from non-passable obstacles by internally simulating movement sequences.

Contribution

The paper introduces a novel visuomotor simulation model for affordance detection, validated on a real robot, with hierarchical training of internal models for reliable discrimination.

Findings

Robot reliably distinguishes corridors from dead ends.

Hierarchical training of internal models improves detection accuracy.

Simulation-based approach enhances real-world applicability.

Abstract

The term "affordance" denotes the behavioral meaning of objects. We propose a cognitive architecture for the detection of affordances in the visual modality. This model is based on the internal simulation of movement sequences. For each movement step, the resulting sensory state is predicted by a forward model, which in turn triggers the generation of a new (simulated) motor command by an inverse model. Thus, a series of mental images in the sensory and in the motor domain is evoked. Starting from a real sensory state, a large number of such sequences is simulated in parallel. Final affordance detection is based on the generated motor commands. We apply this model to a real-world mobile robot which is faced with obstacle arrangements some of which are passable (corridor) and some of which are not (dead ends). The robot's task is to detect the right affordance ("pass-through-able" or "non-pass-through-able"). The required internal models are acquired in a hierarchical training process. Afterwards, the robotic agent is able to distinguish reliably between corridors and dead ends. This real-world result enhances the validity of the proposed mental simulation approach. In addition, we compare several key factors in the simulation process regarding performance and efficiency.