Bin-picking of novel objects through category-agnostic-segmentation: RGB matters

TL;DR

This paper introduces a category-agnostic instance segmentation approach for robotic bin-picking that effectively handles transparent objects and noisy sensors, achieving high accuracy and outperforming existing methods.

Contribution

A novel simulation-trained, domain-randomized instance segmentation framework that generalizes well to real-world bin-picking, including transparent objects.

Findings

Achieved 98% accuracy on opaque objects in real bin-picking.

Outperformed state-of-the-art benchmarks on WISDOM dataset.

Successfully handled transparent and semi-transparent objects.

Abstract

This paper addresses category-agnostic instance segmentation for robotic manipulation, focusing on segmenting objects independent of their class to enable versatile applications like bin-picking in dynamic environments. Existing methods often lack generalizability and object-specific information, leading to grasp failures. We present a novel approach leveraging object-centric instance segmentation and simulation-based training for effective transfer to real-world scenarios. Notably, our strategy overcomes challenges posed by noisy depth sensors, enhancing the reliability of learning. Our solution accommodates transparent and semi-transparent objects which are historically difficult for depth-based grasping methods. Contributions include domain randomization for successful transfer, our collected dataset for warehouse applications, and an integrated framework for efficient bin-picking. Our trained instance segmentation model achieves state-of-the-art performance over WISDOM public benchmark [1] and also over the custom-created dataset. In a real-world challenging bin-picking setup our bin-picking framework method achieves 98% accuracy for opaque objects and 97% accuracy for non-opaque objects, outperforming the state-of-the-art baselines with a greater margin.