Recognizing Objects In-the-wild: Where Do We Stand?

TL;DR

This paper introduces a large-scale, multi-view robotic dataset for object recognition in real-world environments, evaluates deep learning models on it, and discusses the challenges and transferability issues faced in robotic vision.

Contribution

It presents a new robotic dataset capturing real-world challenges and assesses deep learning models' performance and transferability in robotic object recognition.

Findings

Deep models perform well on web images but less so on robotic data.

Robotic object recognition remains a significant challenge.

Transfer learning from web images to robotic data is promising but imperfect.

Abstract

The ability to recognize objects is an essential skill for a robotic system acting in human-populated environments. Despite decades of effort from the robotic and vision research communities, robots are still missing good visual perceptual systems, preventing the use of autonomous agents for real-world applications. The progress is slowed down by the lack of a testbed able to accurately represent the world perceived by the robot in-the-wild. In order to fill this gap, we introduce a large-scale, multi-view object dataset collected with an RGB-D camera mounted on a mobile robot. The dataset embeds the challenges faced by a robot in a real-life application and provides a useful tool for validating object recognition algorithms. Besides describing the characteristics of the dataset, the paper evaluates the performance of a collection of well-established deep convolutional networks on the new dataset and analyzes the transferability of deep representations from Web images to robotic data. Despite the promising results obtained with such representations, the experiments demonstrate that object classification with real-life robotic data is far from being solved. Finally, we provide a comparative study to analyze and highlight the open challenges in robot vision, explaining the discrepancies in the performance.