Are we done with object recognition? The iCub robot's perspective

TL;DR

This paper evaluates current deep learning methods for object recognition on the iCub robot, highlighting benefits, limitations, and open challenges in robotic vision compared to traditional image retrieval tasks.

Contribution

It introduces a new dataset based on human-robot interaction, analyzes knowledge transfer needs, and identifies bottlenecks for deploying deep learning in robotic scenarios.

Findings

Deep learning significantly improves robotic object recognition performance.

Knowledge transfer is essential for effective recognition in robotics.

Major bottlenecks include domain gap and dataset biases.

Abstract

We report on an extensive study of the benefits and limitations of current deep learning approaches to object recognition in robot vision scenarios, introducing a novel dataset used for our investigation. To avoid the biases in currently available datasets, we consider a natural human-robot interaction setting to design a data-acquisition protocol for visual object recognition on the iCub humanoid robot. Analyzing the performance of off-the-shelf models trained off-line on large-scale image retrieval datasets, we show the necessity for knowledge transfer. We evaluate different ways in which this last step can be done, and identify the major bottlenecks affecting robotic scenarios. By studying both object categorization and identification problems, we highlight key differences between object recognition in robotics applications and in image retrieval tasks, for which the considered deep learning approaches have been originally designed. In a nutshell, our results confirm the remarkable improvements yield by deep learning in this setting, while pointing to specific open challenges that need be addressed for seamless deployment in robotics.