ComPC: Completing a 3D Point Cloud with 2D Diffusion Priors
Tianxin Huang, Zhiwen Yan, Yuyang Zhao, Gim Hee Lee
TL;DR
This paper introduces ComPC, a novel test-time method that uses pre-trained 2D diffusion models to complete incomplete 3D point clouds across unseen categories without training, outperforming existing methods.
Contribution
We propose a training-free framework leveraging 2D diffusion priors and Gaussian Splatting for 3D point cloud completion across unseen categories.
Findings
Outperforms existing methods on synthetic and real-world data
Effective in completing diverse object categories
Operates without training on specific object categories
Abstract
3D point clouds directly collected from objects through sensors are often incomplete due to self-occlusion. Conventional methods for completing these partial point clouds rely on manually organized training sets and are usually limited to object categories seen during training. In this work, we propose a test-time framework for completing partial point clouds across unseen categories without any requirement for training. Leveraging point rendering via Gaussian Splatting, we develop techniques of Partial Gaussian Initialization, Zero-shot Fractal Completion, and Point Cloud Extraction that utilize priors from pre-trained 2D diffusion models to infer missing regions and extract uniform completed point clouds. Experimental results on both synthetic and real-world scanned point clouds demonstrate that our approach outperforms existing methods in completing a variety of objects. Our project…
Peer Reviews
Decision·ICLR 2025 Poster
The proposed framework generates complete point clouds that accurately capture the underlying surface, entirely bypassing the need for training on extensive 3D shape categories. By leveraging a 2D diffusion model to synthesize novel views from a specific input viewpoint, it also removes the need for manual text prompts. Furthermore, this approach achieves superior or comparable performance to other test-time frameworks (e.g., SDS-complete) while significantly reducing computational time. Finally
Although this method claims to rely solely on 3D coordinates, as required by the point cloud completion task, it implicitly introduces additional information by incorporating the 3D normal of each point to encode the color map $G_{in}^c$. This additional input signal, which is not utilized by other methods such as PCN or SDS-complete, may introduce an advantage, as evidenced by its significant impact on performance shown in Table 3 and Figure 7. Furthermore, the method heavily depends on the inp
a) The problem of completing partial point clouds in a generalizable way is important, as existing methods mostly yield good results only on in-domain samples. b) 2D/3D priors from diffusion models are well-suited for achieving generalization. c) The paper is well-written, the problem is clearly specified and the solution is well presented.
a) As the incompleteness of the partial point cloud increases, the diffusion model may produce different completion results at each optimization. However, the multi-modal nature of the approach is not analyzed. For a small set of shapes, multiple runs can be performed on the same partial input, and the resulting completions can be evaluated using multi-modal metrics such as TMD, UHD, MMD. b) The experiment in A.5 could be performed on observations with varying levels of incompleteness, rather th
Learning-based methods perform not good when tested out of the domain they were trained on. The strong point of the proposed method is that authors propose a framework that performs out of the training domain as good as in it. Moreover, the model does not require any additional modalities, so the possibility of application is maximal. Lastly, there are empirical experiments done that prove that proposed method outperforms overviewed analogical approaches. The ablation study is well done: this pa
Despite the strengths of the work, there are a number of questions about it. There is a method of missing parts completion in section 3.2. Many side methods like Zero 1-to-3, SDS guidance, PFNet are referenced in this section while there are very few particular discussions that are taken from these works. For example, what is the fractal approach discussion taken from PFNet? The second question is: what is the point of applying differentiable quantization to set the gaussians' opacity? It seems
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Optical Sensing Technologies · Advanced X-ray and CT Imaging · Random lasers and scattering media
MethodsDiffusion · Colorization