Spatio-Temporal Metric-Semantic Mapping for Persistent Orchard Monitoring: Method and Dataset

TL;DR

This paper introduces a 4D spatio-temporal mapping system combining LiDAR and RGB data to accurately monitor orchard growth, enabling precise fruit tracking and size estimation over time.

Contribution

It presents a novel multi-session 4D mapping approach with a fusion and association method, and releases a comprehensive orchard dataset for future research.

Findings

Achieved 96.9% fruit counting accuracy

Reduced data association errors by 23.7%

Estimated fruit size with 1.1 cm error

Abstract

Monitoring orchards at the individual tree or fruit level throughout the growth season is crucial for plant phenotyping and horticultural resource optimization, such as chemical use and yield estimation. We present a 4D spatio-temporal metric-semantic mapping system that integrates multi-session measurements to track fruit growth over time. Our approach combines a LiDAR-RGB fusion module for 3D fruit localization with a 4D fruit association method leveraging positional, visual, and topology information for improved data association precision. Evaluated on real orchard data, our method achieves a 96.9% fruit counting accuracy for 1,790 apples across 60 trees, a mean fruit size estimation error of 1.1 cm, and a 23.7% improvement in 4D data association precision over baselines. We publicly release a multimodal dataset covering five fruit species across their growth seasons at https://4d-metric-semantic-mapping.org/