Sparkle: A Robust and Versatile Representation for Point Cloud based Human Motion Capture

TL;DR

Sparkle introduces a structured, kinematic-geometric representation for point cloud human motion capture, enhancing robustness and generalization in noisy and unstructured data environments.

Contribution

It proposes Sparkle, a novel unified representation combining skeletal and surface features with explicit kinematic-geometric factorization, improving motion capture accuracy and robustness.

Findings

Achieves state-of-the-art accuracy in human motion capture.

Demonstrates superior robustness under noise, occlusion, and domain shifts.

Excels across diverse sensor types and real-world scenarios.

Abstract

Point cloud-based motion capture leverages rich spatial geometry and privacy-preserving sensing, but learning robust representations from noisy, unstructured point clouds remains challenging. Existing approaches face a struggle trade-off between point-based methods (geometrically detailed but noisy) and skeleton-based ones (robust but oversimplified). We address the fundamental challenge: how to construct an effective representation for human motion capture that can balance expressiveness and robustness. In this paper, we propose Sparkle, a structured representation unifying skeletal joints and surface anchors with explicit kinematic-geometric factorization. Our framework, SparkleMotion, learns this representation through hierarchical modules embedding geometric continuity and kinematic constraints. By explicitly disentangling internal kinematic structure from external surface geometry, SparkleMotion achieves state-of-the-art performance not only in accuracy but crucially in robustness and generalization under severe domain shifts, noise, and occlusion. Extensive experiments demonstrate our superiority across diverse sensor types and challenging real-world scenarios.