Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly

TL;DR

This paper explores sim2real transfer learning for point cloud segmentation in industrial robotics, proposing methods to generate synthetic data, address class imbalance, and introducing a novel attention network to improve real-world performance.

Contribution

It introduces a new approach for sim2real transfer in point cloud segmentation, including synthetic data generation, imbalance mitigation strategies, and a novel patch-based attention network.

Findings

Synthetic point cloud data improves real-world segmentation performance.

Multiple strategies effectively address class imbalance.

The proposed attention network enhances model accuracy in industrial scenarios.

Abstract

On robotics computer vision tasks, generating and annotating large amounts of data from real-world for the use of deep learning-based approaches is often difficult or even impossible. A common strategy for solving this problem is to apply simulation-to-reality (sim2real) approaches with the help of simulated scenes. While the majority of current robotics vision sim2real work focuses on image data, we present an industrial application case that uses sim2real transfer learning for point cloud data. We provide insights on how to generate and process synthetic point cloud data in order to achieve better performance when the learned model is transferred to real-world data. The issue of imbalanced learning is investigated using multiple strategies. A novel patch-based attention network is proposed additionally to tackle this problem.