Vision-assisted Avocado Harvesting with Aerial Bimanual Manipulation

TL;DR

This paper presents a novel aerial bimanual robotic system using visual perception and learning to autonomously detect, reach, and harvest avocados, addressing challenges posed by unstructured environments and high-reaching fruits.

Contribution

It introduces a dual-arm UAV with a fixation and gripper arm, enabling efficient harvesting by stabilizing peduncles and executing rotational detachment, a novel approach in aerial fruit harvesting.

Findings

Successful autonomous detection and pose estimation of avocados.

Effective stabilization of peduncles during harvesting.

Demonstrated system efficiency through integrated experiments.

Abstract

Robotic fruit harvesting holds potential in precision agriculture to improve harvesting efficiency. While ground mobile robots are mostly employed in fruit harvesting, certain crops, like avocado trees, cannot be harvested efficiently from the ground alone. This is because of unstructured ground and planting arrangement and high-to-reach fruits. In such cases, aerial robots integrated with manipulation capabilities can pave new ways in robotic harvesting. This paper outlines the design and implementation of a bimanual UAV that employs visual perception and learning to autonomously detect avocados, reach, and harvest them. The dual-arm system comprises a gripper and a fixer arm, to address a key challenge when harvesting avocados: once grasped, a rotational motion is the most efficient way to detach the avocado from the peduncle; however, the peduncle may store elastic energy preventing the avocado from being harvested. The fixer arm aims to stabilize the peduncle, allowing the gripper arm to harvest. The integrated visual perception process enables the detection of avocados and the determination of their pose; the latter is then used to determine target points for a bimanual manipulation planner. Several experiments are conducted to assess the efficacy of each component, and integrated experiments assess the effectiveness of the system.