An End-to-End Robust Point Cloud Semantic Segmentation Network with Single-Step Conditional Diffusion Models

TL;DR

This paper introduces CDSegNet, a novel end-to-end 3D point cloud segmentation network that leverages a conditional-noise framework of diffusion models to improve robustness and achieve state-of-the-art results with single-step inference.

Contribution

The paper proposes CDSegNet, a new diffusion-based network that models noise as a learnable feature generator, enabling efficient single-step semantic segmentation of 3D scenes.

Findings

Outperforms existing methods on indoor and outdoor benchmarks

Exhibits strong noise and sparsity robustness

Achieves state-of-the-art segmentation accuracy

Abstract

Existing conditional Denoising Diffusion Probabilistic Models (DDPMs) with a Noise-Conditional Framework (NCF) remain challenging for 3D scene understanding tasks, as the complex geometric details in scenes increase the difficulty of fitting the gradients of the data distribution (the scores) from semantic labels. This also results in longer training and inference time for DDPMs compared to non-DDPMs. From a different perspective, we delve deeply into the model paradigm dominated by the Conditional Network. In this paper, we propose an end-to-end robust semantic Segmentation Network based on a Conditional-Noise Framework (CNF) of DDPMs, named CDSegNet. Specifically, CDSegNet models the Noise Network (NN) as a learnable noise-feature generator. This enables the Conditional Network (CN) to understand 3D scene semantics under multi-level feature perturbations, enhancing the generalization in unseen scenes. Meanwhile, benefiting from the noise system of DDPMs, CDSegNet exhibits strong noise and sparsity robustness in experiments. Moreover, thanks to CNF, CDSegNet can generate the semantic labels in a single-step inference like non-DDPMs, due to avoiding directly fitting the scores from semantic labels in the dominant network of CDSegNet. On public indoor and outdoor benchmarks, CDSegNet significantly outperforms existing methods, achieving state-of-the-art performance.