Learning to Infer Parameterized Representations of Plants from 3D Scans

TL;DR

This paper introduces a data-driven method using recursive neural networks to infer parameterized 3D plant architectures from scans, enabling multiple plant phenotyping tasks with good generalization from synthetic training data.

Contribution

The authors present a novel approach that combines recursive neural networks and procedural models to reconstruct detailed plant structures from 3D scans, applicable to any binary axial tree plant.

Findings

Achieves competitive results in reconstruction, segmentation, and skeletonization.

Generalizes well from synthetic to real 3D plant scans.

Works effectively on Chenopodium Album plants.

Abstract

Plants frequently contain numerous organs, organized in 3D branching systems defining the plant's architecture. Reconstructing the architecture of plants from unstructured observations is challenging because of self-occlusion and spatial proximity between organs, which are often thin structures. To achieve the challenging task, we propose an approach that allows to infer a parameterized representation of the plant's architecture from a given 3D scan of a plant. In addition to the plant's branching structure, this representation contains parametric information for each plant organ, and can therefore be used directly in a variety of tasks. In this data-driven approach, we train a recursive neural network with virtual plants generated using a procedural model. After training, the network allows to infer a parametric tree-like representation based on an input 3D point cloud. Our method is applicable to any plant that can be represented as binary axial tree. We quantitatively evaluate our approach on Chenopodium Album plants on reconstruction, segmentation and skeletonization, which are important problems in plant phenotyping. In addition to carrying out several tasks at once, our method achieves results on-par with strong baselines for each task. We apply our method, trained exclusively on synthetic data, to 3D scans and show that it generalizes well.