3D-PSRNet: Part Segmented 3D Point Cloud Reconstruction From a Single Image

TL;DR

This paper introduces a method for reconstructing part-segmented 3D point clouds from a single image by jointly training for reconstruction and segmentation, leading to improved performance in both tasks.

Contribution

It proposes a joint training framework with a novel location-aware segmentation loss to enhance 3D reconstruction and segmentation accuracy from a single image.

Findings

Joint training improves both reconstruction and segmentation performance.

The location-aware segmentation loss enhances part fidelity.

The approach outperforms existing methods on ShapeNet dataset.

Abstract

We propose a mechanism to reconstruct part annotated 3D point clouds of objects given just a single input image. We demonstrate that jointly training for both reconstruction and segmentation leads to improved performance in both the tasks, when compared to training for each task individually. The key idea is to propagate information from each task so as to aid the other during the training procedure. Towards this end, we introduce a location-aware segmentation loss in the training regime. We empirically show the effectiveness of the proposed loss in generating more faithful part reconstructions while also improving segmentation accuracy. We thoroughly evaluate the proposed approach on different object categories from the ShapeNet dataset to obtain improved results in reconstruction as well as segmentation.