AHPPEBot: Autonomous Robot for Tomato Harvesting based on Phenotyping and Pose Estimation

TL;DR

AHPPEBot is an autonomous tomato harvesting robot that uses phenotyping and pose estimation techniques to improve success rates and reduce crop damage, demonstrating promising results in greenhouse experiments.

Contribution

The paper introduces a novel robot combining multi-task YOLOv5 and deep learning pose estimation for efficient autonomous tomato harvesting.

Findings

Harvesting success rate of 86.67%

Average harvest time of 32.46 seconds

Robust performance in commercial greenhouses

Abstract

To address the limitations inherent to conventional automated harvesting robots specifically their suboptimal success rates and risk of crop damage, we design a novel bot named AHPPEBot which is capable of autonomous harvesting based on crop phenotyping and pose estimation. Specifically, In phenotyping, the detection, association, and maturity estimation of tomato trusses and individual fruits are accomplished through a multi-task YOLOv5 model coupled with a detection-based adaptive DBScan clustering algorithm. In pose estimation, we employ a deep learning model to predict seven semantic keypoints on the pedicel. These keypoints assist in the robot's path planning, minimize target contact, and facilitate the use of our specialized end effector for harvesting. In autonomous tomato harvesting experiments conducted in commercial greenhouses, our proposed robot achieved a harvesting success rate of 86.67%, with an average successful harvest time of 32.46 s, showcasing its continuous and robust harvesting capabilities. The result underscores the potential of harvesting robots to bridge the labor gap in agriculture.