PointDico: Contrastive 3D Representation Learning Guided by Diffusion Models

TL;DR

PointDico introduces a novel 3D representation learning method that combines diffusion and contrastive models via knowledge distillation, achieving state-of-the-art results on key 3D benchmarks.

Contribution

The paper proposes PointDico, a new model integrating diffusion and contrastive learning for 3D data, with a hierarchical generator and dual-channel design for improved feature extraction.

Findings

Achieves 94.32% accuracy on ScanObjectNN

Attains 86.5% Inst. mIoU on ShapeNetPart

Outperforms previous state-of-the-art methods

Abstract

Self-supervised representation learning has shown significant improvement in Natural Language Processing and 2D Computer Vision. However, existing methods face difficulties in representing 3D data because of its unordered and uneven density. Through an in-depth analysis of mainstream contrastive and generative approaches, we find that contrastive models tend to suffer from overfitting, while 3D Mask Autoencoders struggle to handle unordered point clouds. This motivates us to learn 3D representations by sharing the merits of diffusion and contrast models, which is non-trivial due to the pattern difference between the two paradigms. In this paper, we propose \textit{PointDico}, a novel model that seamlessly integrates these methods. \textit{PointDico} learns from both denoising generative modeling and cross-modal contrastive learning through knowledge distillation, where the diffusion model serves as a guide for the contrastive model. We introduce a hierarchical pyramid conditional generator for multi-scale geometric feature extraction and employ a dual-channel design to effectively integrate local and global contextual information. \textit{PointDico} achieves a new state-of-the-art in 3D representation learning, \textit{e.g.}, \textbf{94.32\%} accuracy on ScanObjectNN, \textbf{86.5\%} Inst. mIoU on ShapeNetPart.