A Hybrid Generative and Discriminative PointNet on Unordered Point Sets

TL;DR

This paper introduces GDPNet, a hybrid model that combines generative and discriminative capabilities for point cloud analysis, achieving high-quality generation and classification within a single network.

Contribution

It extends the Joint Energy-based Model framework to point clouds, enabling a unified approach for both generation and classification tasks.

Findings

GDPNet achieves state-of-the-art point cloud generation quality.

It maintains strong classification performance comparable to modern PointNet.

The model demonstrates effective joint training for generation and discrimination.

Abstract

As point cloud provides a natural and flexible representation usable in myriad applications (e.g., robotics and self-driving cars), the ability to synthesize point clouds for analysis becomes crucial. Recently, Xie et al. propose a generative model for unordered point sets in the form of an energy-based model (EBM). Despite the model achieving an impressive performance for point cloud generation, one separate model needs to be trained for each category to capture the complex point set distributions. Besides, their method is unable to classify point clouds directly and requires additional fine-tuning for classification. One interesting question is: Can we train a single network for a hybrid generative and discriminative model of point clouds? A similar question has recently been answered in the affirmative for images, introducing the framework of Joint Energy-based Model (JEM), which achieves high performance in image classification and generation simultaneously. This paper proposes GDPNet, the first hybrid Generative and Discriminative PointNet that extends JEM for point cloud classification and generation. Our GDPNet retains strong discriminative power of modern PointNet classifiers, while generating point cloud samples rivaling state-of-the-art generative approaches.