SVDFormer: Complementing Point Cloud via Self-view Augmentation and Self-structure Dual-generator

TL;DR

SVDFormer introduces a novel point cloud completion network that leverages multi-view depth images and a dual-generator refinement module to accurately reconstruct global shapes and detailed local structures without extra paired data.

Contribution

The paper proposes SVDFormer, a new network combining self-view fusion and a dual-generator refinement to improve point cloud completion without relying on additional paired information.

Findings

Achieves state-of-the-art results on benchmark datasets.

Effectively reconstructs global shapes from incomplete point clouds.

Produces high-accuracy local structures with learned priors.

Abstract

In this paper, we propose a novel network, SVDFormer, to tackle two specific challenges in point cloud completion: understanding faithful global shapes from incomplete point clouds and generating high-accuracy local structures. Current methods either perceive shape patterns using only 3D coordinates or import extra images with well-calibrated intrinsic parameters to guide the geometry estimation of the missing parts. However, these approaches do not always fully leverage the cross-modal self-structures available for accurate and high-quality point cloud completion. To this end, we first design a Self-view Fusion Network that leverages multiple-view depth image information to observe incomplete self-shape and generate a compact global shape. To reveal highly detailed structures, we then introduce a refinement module, called Self-structure Dual-generator, in which we incorporate learned shape priors and geometric self-similarities for producing new points. By perceiving the incompleteness of each point, the dual-path design disentangles refinement strategies conditioned on the structural type of each point. SVDFormer absorbs the wisdom of self-structures, avoiding any additional paired information such as color images with precisely calibrated camera intrinsic parameters. Comprehensive experiments indicate that our method achieves state-of-the-art performance on widely-used benchmarks. Code will be available at https://github.com/czvvd/SVDFormer.