Addressing the Sim2Real Gap in Robotic 3D Object Classification

TL;DR

This paper investigates the challenge of transferring 3D object classification models trained on CAD models to real reconstructed objects in robotics, proposing a part-based graph convolution approach to improve robustness and accuracy.

Contribution

It introduces a novel part sampling and graph convolution method that enhances cross-domain 3D object classification from CAD models to real-world data.

Findings

Significant performance improvement over baseline methods.

Robustness achieved through part-based sampling under transformations.

Effective transfer from CAD models to reconstructed real objects.

Abstract

Object classification with 3D data is an essential component of any scene understanding method. It has gained significant interest in a variety of communities, most notably in robotics and computer graphics. While the advent of deep learning has progressed the field of 3D object classification, most work using this data type are solely evaluated on CAD model datasets. Consequently, current work does not address the discrepancies existing between real and artificial data. In this work, we examine this gap in a robotic context by specifically addressing the problem of classification when transferring from artificial CAD models to real reconstructed objects. This is performed by training on ModelNet (CAD models) and evaluating on ScanNet (reconstructed objects). We show that standard methods do not perform well in this task. We thus introduce a method that carefully samples object parts that are reproducible under various transformations and hence robust. Using graph convolution to classify the composed graph of parts, our method significantly improves upon the baseline.