Robotic Grasping of Harvested Tomato Trusses Using Vision and Online Learning

TL;DR

This paper presents a vision-based robotic grasping system with online learning for efficiently harvesting stacked tomato trusses in cluttered environments, achieving high success rates without tactile sensors.

Contribution

It introduces a deep learning vision system combined with an online learning grasp ranking algorithm for reliable, sensor-free robotic grasping of harvested tomato trusses.

Findings

100% success in clearing trusses from a pile

93% of trusses grasped on first attempt

7% required multiple attempts

Abstract

Currently, truss tomato weighing and packaging require significant manual work. The main obstacle to automation lies in the difficulty of developing a reliable robotic grasping system for already harvested trusses. We propose a method to grasp trusses that are stacked in a crate with considerable clutter, which is how they are commonly stored and transported after harvest. The method consists of a deep learning-based vision system to first identify the individual trusses in the crate and then determine a suitable grasping location on the stem. To this end, we have introduced a grasp pose ranking algorithm with online learning capabilities. After selecting the most promising grasp pose, the robot executes a pinch grasp without needing touch sensors or geometric models. Lab experiments with a robotic manipulator equipped with an eye-in-hand RGB-D camera showed a 100% clearance rate when tasked to pick all trusses from a pile. 93% of the trusses were successfully grasped on the first try, while the remaining 7% required more attempts.