Graph-based Cluttered Scene Generation and Interactive Exploration using Deep Reinforcement Learning

TL;DR

This paper presents a deep reinforcement learning framework for generating and exploring cluttered scenes, enabling robots to discover hidden objects efficiently in structured environments like kitchens.

Contribution

It introduces a novel scene grammar and a GNN-based scene generation method, along with an exploration policy that generalizes to real-world cluttered scenes.

Findings

Agents outperform baselines in object discovery

Effective sim-to-real transfer demonstrated on UR10 robot

Scene generation produces diverse stable cluttered scenes

Abstract

We introduce a novel method to teach a robotic agent to interactively explore cluttered yet structured scenes, such as kitchen pantries and grocery shelves, by leveraging the physical plausibility of the scene. We propose a novel learning framework to train an effective scene exploration policy to discover hidden objects with minimal interactions. First, we define a novel scene grammar to represent structured clutter. Then we train a Graph Neural Network (GNN) based Scene Generation agent using deep reinforcement learning (deep RL), to manipulate this Scene Grammar to create a diverse set of stable scenes, each containing multiple hidden objects. Given such cluttered scenes, we then train a Scene Exploration agent, using deep RL, to uncover hidden objects by interactively rearranging the scene. We show that our learned agents hide and discover significantly more objects than the baselines. We present quantitative results that prove the generalization capabilities of our agents. We also demonstrate sim-to-real transfer by successfully deploying the learned policy on a real UR10 robot to explore real-world cluttered scenes. The supplemental video can be found at https://www.youtube.com/watch?v=T2Jo7wwaXss.