A comprehensive control architecture for semi-autonomous dual-arm robots in agriculture settings

TL;DR

This paper presents a hierarchical control architecture for a dual-arm mobile robot in agriculture, enabling complex, semi-autonomous harvesting tasks with integrated perception, control, and human interaction.

Contribution

It introduces a comprehensive HQP-based control framework capable of handling multiple constraints and interaction forces for semi-autonomous vineyard harvesting.

Findings

Successful validation in laboratory and vineyard environments.

Effective handling of constraints and collision avoidance.

Enabling semi-autonomous operation with human assistance.

Abstract

The adoption of mobile robotic platforms in complex environments, such as agricultural settings, requires these systems to exhibit a flexible yet effective architecture that integrates perception and control. In such scenarios, several tasks need to be accomplished simultaneously, ranging from managing robot limits to performing operational tasks and handling human inputs. The purpose of this paper is to present a comprehensive control architecture for achieving complex tasks such as robotized harvesting in vineyards within the framework of the European project CANOPIES. In detail, a 16-DOF dual-arm mobile robot is employed, controlled via a Hierarchical Quadratic Programming (HQP) approach capable of handling both equality and inequality constraints at various priorities to harvest grape bunches selected by the perception system developed within the project. Furthermore, given the complexity of the scenario and the uncertainty in the perception system, which could potentially lead to collisions with the environment, the handling of interaction forces is necessary. Remarkably, this was achieved using the same HQP framework. This feature is further leveraged to enable semi-autonomous operations, allowing a human operator to assist the robotic counterpart in completing harvesting tasks. Finally, the obtained results are validated through extensive testing conducted first in a laboratory environment to prove individual functionalities, then in a real vineyard, encompassing both autonomous and semi-autonomous grape harvesting operations.