CF-PRNet: Coarse-to-Fine Prototype Refining Network for Point Cloud Completion and Reconstruction

TL;DR

CF-PRNet is a novel coarse-to-fine network that accurately reconstructs 3D fruit shapes from partial RGB-D views, leveraging high-resolution training data for real-time agricultural applications.

Contribution

The paper introduces CF-PRNet, a new method that refines 3D prototypes progressively for accurate point cloud completion from limited input data.

Findings

Achieved a Chamfer Distance of 3.78

Won first place in the Sweet Peppers Challenge

Demonstrated high accuracy in 3D fruit reconstruction

Abstract

In modern agriculture, precise monitoring of plants and fruits is crucial for tasks such as high-throughput phenotyping and automated harvesting. This paper addresses the challenge of reconstructing accurate 3D shapes of fruits from partial views, which is common in agricultural settings. We introduce CF-PRNet, a coarse-to-fine prototype refining network, leverages high-resolution 3D data during the training phase but requires only a single RGB-D image for real-time inference. Our approach begins by extracting the incomplete point cloud data that constructed from a partial view of a fruit with a series of convolutional blocks. The extracted features inform the generation of scaling vectors that refine two sequentially constructed 3D mesh prototypes - one coarse and one fine-grained. This progressive refinement facilitates the detailed completion of the final point clouds, achieving detailed and accurate reconstructions. CF-PRNet demonstrates excellent performance metrics with a Chamfer Distance of 3.78, an F1 Score of 66.76%, a Precision of 56.56%, and a Recall of 85.31%, and win the first place in the Shape Completion and Reconstruction of Sweet Peppers Challenge.