A New Framework for Registration of Semantic Point Clouds from Stereo and RGB-D Cameras

TL;DR

This paper introduces a novel nonparametric framework for registering semantic point clouds from stereo and RGB-D cameras, integrating geometric and semantic data without explicit data association, and demonstrating superior performance over existing methods.

Contribution

The paper presents a new nonparametric registration framework that combines geometric and semantic information in a unified optimization process, without requiring explicit data association.

Findings

Outperforms state-of-the-art registration methods on public datasets

Provides an analytical gradient-based optimization approach

Includes an open-source GPU implementation

Abstract

This paper reports on a novel nonparametric rigid point cloud registration framework that jointly integrates geometric and semantic measurements such as color or semantic labels into the alignment process and does not require explicit data association. The point clouds are represented as nonparametric functions in a reproducible kernel Hilbert space. The alignment problem is formulated as maximizing the inner product between two functions, essentially a sum of weighted kernels, each of which exploits the local geometric and semantic features. As a result of the continuous models, analytical gradients can be computed, and a local solution can be obtained by optimization over the rigid body transformation group. Besides, we present a new point cloud alignment metric that is intrinsic to the proposed framework and takes into account geometric and semantic information. The evaluations using publicly available stereo and RGB-D datasets show that the proposed method outperforms state-of-the-art outdoor and indoor frame-to-frame registration methods. An open-source GPU implementation is also provided.