Leveraging SE(3) Equivariance for Learning 3D Geometric Shape Assembly

TL;DR

This paper introduces a novel approach using SE(3) equivariance to improve 3D geometric shape assembly by disentangling shape pose and considering multi-part correlations, leading to better assembly performance.

Contribution

It proposes leveraging SE(3) equivariance for shape pose disentanglement and multi-part correlation modeling in 3D shape assembly, advancing beyond single-object representations.

Findings

SE(3) equivariance significantly improves shape assembly accuracy.

Model considering multi-part correlations outperforms single-part approaches.

Experimental results validate the effectiveness of the proposed method.

Abstract

Shape assembly aims to reassemble parts (or fragments) into a complete object, which is a common task in our daily life. Different from the semantic part assembly (e.g., assembling a chair's semantic parts like legs into a whole chair), geometric part assembly (e.g., assembling bowl fragments into a complete bowl) is an emerging task in computer vision and robotics. Instead of semantic information, this task focuses on geometric information of parts. As the both geometric and pose space of fractured parts are exceptionally large, shape pose disentanglement of part representations is beneficial to geometric shape assembly. In our paper, we propose to leverage SE(3) equivariance for such shape pose disentanglement. Moreover, while previous works in vision and robotics only consider SE(3) equivariance for the representations of single objects, we move a step forward and propose leveraging SE(3) equivariance for representations considering multi-part correlations, which further boosts the performance of the multi-part assembly. Experiments demonstrate the significance of SE(3) equivariance and our proposed method for geometric shape assembly. Project page: https://crtie.github.io/SE-3-part-assembly/