Imagine with the Teacher: Complete Shape in a Multi-View Distillation Way

TL;DR

This paper introduces VD-PCN, a novel multi-view distillation approach for 3D point cloud completion that leverages 2D image processing techniques and knowledge transfer to improve shape reconstruction from partial data.

Contribution

It proposes a new multi-view distillation framework for point cloud completion, effectively transferring knowledge from 2D views to 3D shape reconstruction.

Findings

Outperforms existing methods on PCN, ShapeNet55/34, and MVP datasets.

Demonstrates significant improvements in both quantitative metrics and qualitative shape quality.

Validates the effectiveness of multi-view distillation in 3D shape completion.

Abstract

Point cloud completion aims to recover the completed 3D shape of an object from its partial observation caused by occlusion, sensor's limitation, noise, etc. When some key semantic information is lost in the incomplete point cloud, the neural network needs to infer the missing part based on the input information. Intuitively we would apply an autoencoder architecture to solve this kind of problem, which take the incomplete point cloud as input and is supervised by the ground truth. This process that develops model's imagination from incomplete shape to complete shape is done automatically in the latent space. But the knowledge for mapping from incomplete to complete still remains dark and could be further explored. Motivated by the knowledge distillation's teacher-student learning strategy, we design a knowledge transfer way for completing 3d shape. In this work, we propose a novel View Distillation Point Completion Network (VD-PCN), which solve the completion problem by a multi-view distillation way. The design methodology fully leverages the orderliness of 2d pixels, flexibleness of 2d processing and powerfulness of 2d network. Extensive evaluations on PCN, ShapeNet55/34, and MVP datasets confirm the effectiveness of our design and knowledge transfer strategy, both quantitatively and qualitatively. Committed to facilitate ongoing research, we will make our code publicly available.