LeafTrackNet: A Deep Learning Framework for Robust Leaf Tracking in Top-Down Plant Phenotyping

TL;DR

LeafTrackNet is a novel deep learning framework designed for accurate and robust leaf tracking in complex crop scenes, supported by a large-scale dataset, CanolaTrack, to advance plant phenotyping research.

Contribution

The paper introduces LeafTrackNet, combining a YOLOv10 detector with MobileNetV3 embeddings, and presents CanolaTrack, the largest dataset for leaf tracking in canola plants under realistic conditions.

Findings

LeafTrackNet achieves a 9% higher HOTA score than existing methods.

CanolaTrack contains 5,704 images with 31,840 annotated leaves.

The framework outperforms plant-specific and generic MOT baselines.

Abstract

High resolution phenotyping at the level of individual leaves offers fine-grained insights into plant development and stress responses. However, the full potential of accurate leaf tracking over time remains largely unexplored due to the absence of robust tracking methods-particularly for structurally complex crops such as canola. Existing plant-specific tracking methods are typically limited to small-scale species or rely on constrained imaging conditions. In contrast, generic multi-object tracking (MOT) methods are not designed for dynamic biological scenes. Progress in the development of accurate leaf tracking models has also been hindered by a lack of large-scale datasets captured under realistic conditions. In this work, we introduce CanolaTrack, a new benchmark dataset comprising 5,704 RGB images with 31,840 annotated leaf instances spanning the early growth stages of 184 canola plants. To enable accurate leaf tracking over time, we introduce LeafTrackNet, an efficient framework that combines a YOLOv10-based leaf detector with a MobileNetV3-based embedding network. During inference, leaf identities are maintained over time through an embedding-based memory association strategy. LeafTrackNet outperforms both plant-specific trackers and state-of-the-art MOT baselines, achieving a 9% HOTA improvement on CanolaTrack. With our work we provide a new standard for leaf-level tracking under realistic conditions and we provide CanolaTrack - the largest dataset for leaf tracking in agriculture crops, which will contribute to future research in plant phenotyping. Our code and dataset are publicly available at https://github.com/shl-shawn/LeafTrackNet.