Object Finding in Cluttered Scenes Using Interactive Perception

TL;DR

This paper presents a reinforcement learning approach for interactive perception in robotics, enabling efficient object search in cluttered scenes through active exploration, trained in simulation and successfully transferred to real-world scenarios.

Contribution

It introduces a reinforcement learning-based system for scene exploration and object search that eliminates the need for predefined heuristics and explicit interaction models.

Findings

Achieves over 88% success rate in real-world object search

Transfers smoothly from simulation to real-world environments

Outperforms baseline methods in efficiency and accuracy

Abstract

Object finding in clutter is a skill that requires perception of the environment and in many cases physical interaction. In robotics, interactive perception defines a set of algorithms that leverage actions to improve the perception of the environment, and vice versa use perception to guide the next action. Scene interactions are difficult to model, therefore, most of the current systems use predefined heuristics. This limits their ability to efficiently search for the target object in a complex environment. In order to remove heuristics and the need for explicit models of the interactions, in this work we propose a reinforcement learning based active and interactive perception system for scene exploration and object search. We evaluate our work both in simulated and in real-world experiments using a robotic manipulator equipped with an RGB and a depth camera, and compare our system to two baselines. The results indicate that our approach, trained in simulation only, transfers smoothly to reality and can solve the object finding task efficiently and with more than 88% success rate.