Detection of On-Ground Chestnuts Using Artificial Intelligence Toward Automated Picking

TL;DR

This study evaluates various state-of-the-art object detection models for accurately identifying chestnuts on orchard floors under complex environmental conditions, aiming to develop cost-effective automated harvesting solutions.

Contribution

It provides a comprehensive comparison of 29 real-time object detectors, identifying YOLOv12m as the most accurate model for chestnut detection in challenging outdoor environments.

Findings

YOLOv12m achieved 95.1% mAP@0.5 accuracy.

RT-DETRv2-R101 achieved 91.1% mAP@0.5.

YOLO models outperform RT-DETR models in accuracy and inference speed.

Abstract

Traditional mechanized chestnut harvesting is too costly for small producers, non-selective, and prone to damaging nuts. Accurate, reliable detection of chestnuts on the orchard floor is crucial for developing low-cost, vision-guided automated harvesting technology. However, developing a reliable chestnut detection system faces challenges in complex environments with shading, varying natural light conditions, and interference from weeds, fallen leaves, stones, and other foreign on-ground objects, which have remained unaddressed. This study collected 319 images of chestnuts on the orchard floor, containing 6524 annotated chestnuts. A comprehensive set of 29 state-of-the-art real-time object detectors, including 14 in the YOLO (v11-13) and 15 in the RT-DETR (v1-v4) families at varied model scales, was systematically evaluated through replicated modeling experiments for chestnut detection. Experimental results show that the YOLOv12m model achieves the best mAP@0.5 of 95.1% among all the evaluated models, while the RT-DETRv2-R101 was the most accurate variant among RT-DETR models, with mAP@0.5 of 91.1%. In terms of mAP@[0.5:0.95], the YOLOv11x model achieved the best accuracy of 80.1%. All models demonstrate significant potential for real-time chestnut detection, and YOLO models outperformed RT-DETR models in terms of both detection accuracy and inference, making them better suited for on-board deployment. Both the dataset and software programs in this study have been made publicly available at https://github.com/AgFood-Sensing-and-Intelligence-Lab/ChestnutDetection.