PointSea: Point Cloud Completion via Self-structure Augmentation

TL;DR

PointSea introduces a novel self-structure augmentation approach for point cloud completion, leveraging multi-view depth images and a dual-generator for detailed shape refinement, achieving superior results on standard benchmarks.

Contribution

The paper presents a new global-to-local point cloud completion framework that uses self-projected depth images and a dual-path generator for detailed shape reconstruction.

Findings

Outperforms existing methods on benchmark datasets

Effectively captures global shape and local details

Adapts refinement strategies based on point structural types

Abstract

Point cloud completion is a fundamental yet not well-solved problem in 3D vision. Current approaches often rely on 3D coordinate information and/or additional data (e.g., images and scanning viewpoints) to fill in missing parts. Unlike these methods, we explore self-structure augmentation and propose PointSea for global-to-local point cloud completion. In the global stage, consider how we inspect a defective region of a physical object, we may observe it from various perspectives for a better understanding. Inspired by this, PointSea augments data representation by leveraging self-projected depth images from multiple views. To reconstruct a compact global shape from the cross-modal input, we incorporate a feature fusion module to fuse features at both intra-view and inter-view levels. In the local stage, to reveal highly detailed structures, we introduce a point generator called the self-structure dual-generator. This generator integrates both learned shape priors and geometric self-similarities for shape refinement. Unlike existing efforts that apply a unified strategy for all points, our dual-path design adapts refinement strategies conditioned on the structural type of each point, addressing the specific incompleteness of each point. Comprehensive experiments on widely-used benchmarks demonstrate that PointSea effectively understands global shapes and generates local details from incomplete input, showing clear improvements over existing methods.