Point Cloud Registration of non-rigid objects in sparse 3D Scans with applications in Mixed Reality

TL;DR

This paper presents a robust method for registering non-rigid objects in sparse 3D scans, specifically for applications in Mixed Reality, addressing challenges like noise, partial data, and object deformations.

Contribution

The paper introduces an efficient registration workflow tailored for non-rigid objects with joint-based deformations in mixed reality scenarios.

Findings

Effective registration on real-world Hololens 2 data

Robustness to noise and partial scans

Applicable to objects with joint-based deformations

Abstract

Point Cloud Registration is the problem of aligning the corresponding points of two 3D point clouds referring to the same object. The challenges include dealing with noise and partial match of real-world 3D scans. For non-rigid objects, there is an additional challenge of accounting for deformations in the object shape that happen to the object in between the two 3D scans. In this project, we study the problem of non-rigid point cloud registration for use cases in the Augmented/Mixed Reality domain. We focus our attention on a special class of non-rigid deformations that happen in rigid objects with parts that move relative to one another about joints, for example, robots with hands and machines with hinges. We propose an efficient and robust point-cloud registration workflow for such objects and evaluate it on real-world data collected using Microsoft Hololens 2, a leading Mixed Reality device.