# Deep Affordance-grounded Sensorimotor Object Recognition

## TL;DR

This paper introduces a deep learning approach inspired by neurobiological principles to fuse appearance and affordance information for improved sensorimotor object recognition, validated on a new RGB-D dataset.

## Contribution

It pioneers the integration of deep neural networks with neuro-inspired architectures for sensorimotor object recognition, surpassing previous methods.

## Key findings

- Up to 29% error reduction with affordance information
- Development of neuro-biologically inspired deep architectures
- Evaluation on a new large RGB-D dataset

## Abstract

It is well-established by cognitive neuroscience that human perception of objects constitutes a complex process, where object appearance information is combined with evidence about the so-called object "affordances", namely the types of actions that humans typically perform when interacting with them. This fact has recently motivated the "sensorimotor" approach to the challenging task of automatic object recognition, where both information sources are fused to improve robustness. In this work, the aforementioned paradigm is adopted, surpassing current limitations of sensorimotor object recognition research. Specifically, the deep learning paradigm is introduced to the problem for the first time, developing a number of novel neuro-biologically and neuro-physiologically inspired architectures that utilize state-of-the-art neural networks for fusing the available information sources in multiple ways. The proposed methods are evaluated using a large RGB-D corpus, which is specifically collected for the task of sensorimotor object recognition and is made publicly available. Experimental results demonstrate the utility of affordance information to object recognition, achieving an up to 29% relative error reduction by its inclusion.