Zero-Shot Sim-to-Real Reinforcement Learning for Fruit Harvesting

TL;DR

This paper develops a sim-to-real reinforcement learning pipeline for autonomous strawberry harvesting, combining simulation with domain randomization and deep RL to enable effective transfer to real robots.

Contribution

It introduces a comprehensive sim-to-real pipeline using a custom Mujoco environment and a novel RL algorithm for fruit harvesting tasks.

Findings

Successful transfer from simulation to real robot

Effective domain randomization techniques used

Promising performance in real laboratory environment

Abstract

This paper presents a comprehensive sim-to-real pipeline for autonomous strawberry picking from dense clusters using a Franka Panda robot. Our approach leverages a custom Mujoco simulation environment that integrates domain randomization techniques. In this environment, a deep reinforcement learning agent is trained using the dormant ratio minimization algorithm. The proposed pipeline bridges low-level control with high-level perception and decision making, demonstrating promising performance in both simulation and in a real laboratory environment, laying the groundwork for successful transfer to real-world autonomous fruit harvesting.