PDT: Point Distribution Transformation with Diffusion Models

TL;DR

PDT introduces a diffusion model-based framework that transforms unstructured point clouds into semantically meaningful and structured point distributions, enhancing 3D shape understanding and processing.

Contribution

It presents a novel diffusion model architecture and learning strategy for transforming point clouds into structured, semantically meaningful distributions, a largely unexplored area.

Findings

Successfully transforms point clouds into structured outputs

Captures geometric and semantic features effectively

Applicable to various 3D geometry processing tasks

Abstract

Point-based representations have consistently played a vital role in geometric data structures. Most point cloud learning and processing methods typically leverage the unordered and unconstrained nature to represent the underlying geometry of 3D shapes. However, how to extract meaningful structural information from unstructured point cloud distributions and transform them into semantically meaningful point distributions remains an under-explored problem. We present PDT, a novel framework for point distribution transformation with diffusion models. Given a set of input points, PDT learns to transform the point set from its original geometric distribution into a target distribution that is semantically meaningful. Our method utilizes diffusion models with novel architecture and learning strategy, which effectively correlates the source and the target distribution through a denoising process. Through extensive experiments, we show that our method successfully transforms input point clouds into various forms of structured outputs - ranging from surface-aligned keypoints, and inner sparse joints to continuous feature lines. The results showcase our framework's ability to capture both geometric and semantic features, offering a powerful tool for various 3D geometry processing tasks where structured point distributions are desired. Code will be available at this link: https://github.com/shanemankiw/PDT.