Simultaneous Localization, Mapping, and Manipulation for Unsupervised Object Discovery

TL;DR

This paper introduces an unsupervised system that enables a robot to discover, track, and reconstruct objects in 3D using RGBD cameras, combining appearance, structure, and motion cues for improved object modeling.

Contribution

It presents a novel unsupervised framework integrating dense SLAM with object discovery and manipulation, including a new spatio-temporal super-pixel approach for better object candidate quality.

Findings

Spatio-temporal super-pixels outperform other methods in object discovery.

The system successfully discovers and reconstructs unknown objects in real-time.

Experimental validation with a Baxter robot demonstrates robustness and accuracy.

Abstract

We present an unsupervised framework for simultaneous appearance-based object discovery, detection, tracking and reconstruction using RGBD cameras and a robot manipulator. The system performs dense 3D simultaneous localization and mapping concurrently with unsupervised object discovery. Putative objects that are spatially and visually coherent are manipulated by the robot to gain additional motion-cues. The robot uses appearance alone, followed by structure and motion cues, to jointly discover, verify, learn and improve models of objects. Induced motion segmentation reinforces learned models which are represented implicitly as 2D and 3D level sets to capture both shape and appearance. We compare three different approaches for appearance-based object discovery and find that a novel form of spatio-temporal super-pixels gives the highest quality candidate object models in terms of precision and recall. Live experiments with a Baxter robot demonstrate a holistic pipeline capable of automatic discovery, verification, detection, tracking and reconstruction of unknown objects.