Unknown Object Segmentation through Domain Adaptation

TL;DR

This paper introduces a domain adaptation framework using GANs to transfer object segmentation models from simulation to real-world robotic grasping tasks, reducing the need for extensive real-world data.

Contribution

It presents a novel sim-to-real approach with a GAN-based domain adaptation method for unknown object segmentation in robotic grasping.

Findings

Segmentation models trained in simulation perform well on real-world data.

The GAN-based domain adaptation reduces the reality gap effectively.

Experimental results validate the approach in bin-picking scenarios.

Abstract

The ability to segment unknown objects in cluttered scenes has a profound impact on robot grasping. The rise of deep learning has greatly transformed the pipeline of robotic grasping from model-based approach to data-driven stream, which generally requires a large scale of grasping data either collected in simulation or from real-world examples. In this paper, we proposed a sim-to-real framework to transfer the object segmentation model learned in simulation to the real-world. First, data samples are collected in simulation, including RGB, 6D pose, and point cloud. Second, we also present a GAN-based unknown object segmentation method through domain adaptation, which consists of an image translation module and an image segmentation module. The image translation module is used to shorten the reality gap and the segmentation module is responsible for the segmentation mask generation. We used the above method to perform segmentation experiments on unknown objects in a bin-picking scenario. Finally, the experimental result shows that the segmentation model learned in simulation can be used for real-world data segmentation.