PointVDP: Learning View-Dependent Projection by Fireworks Rays for 3D Point Cloud Segmentation

TL;DR

PointVDP introduces a view-dependent projection method for 3D point cloud segmentation that adapts projections to spatial geometry, enhancing efficiency and semantic accuracy with minimal computational overhead.

Contribution

The paper presents a novel view-dependent projection framework that generates data-driven, adaptive projections for improved 3D point cloud segmentation.

Findings

Achieves competitive results on S3DIS and ScanNet benchmarks.

Provides a resource-efficient segmentation method with marginal computation costs.

Enhances point awareness through adaptive, informative projections.

Abstract

In this paper, we propose view-dependent projection (VDP) to facilitate point cloud segmentation, designing efficient 3D-to-2D mapping that dynamically adapts to the spatial geometry from view variations. Existing projection-based methods leverage view-independent projection in complex scenes, relying on straight lines to generate direct rays or upward curves to reduce occlusions. However, their view independence provides projection rays that are limited to pre-defined parameters by human settings, restricting point awareness and failing to capture sufficient projection diversity across different view planes. Although multiple projections per view plane are commonly used to enhance spatial variety, the projected redundancy leads to excessive computational overhead and inefficiency in image processing. To address these limitations, we design a framework of VDP to generate data-driven projections from 3D point distributions, producing highly informative single-image inputs by predicting rays inspired by the adaptive behavior of fireworks. In addition, we construct color regularization to optimize the framework, which emphasizes essential features within semantic pixels and suppresses the non-semantic features within black pixels, thereby maximizing 2D space utilization in a projected image. As a result, our approach, PointVDP, develops lightweight projections in marginal computation costs. Experiments on S3DIS and ScanNet benchmarks show that our approach achieves competitive results, offering a resource-efficient solution for semantic understanding.