Render-in-the-loop aerial robotics simulator: Case Study on Yield Estimation in Indoor Agriculture

TL;DR

This paper presents a realistic simulation environment combining ROS-compatible physics simulation with Blender's rendering engine, applied to a case study on aerial robotic yield estimation in indoor agriculture.

Contribution

It introduces a novel simulation pipeline that integrates physics and realistic rendering for sim-to-real transfer in indoor robotic farming applications.

Findings

Successful simulation of RGB-D images for indoor farming

Effective deep learning-based pepper detection and counting

Demonstrated potential for sim-to-real transfer in robotic yield estimation

Abstract

Inspired by recent promising results in sim-to-real transfer in deep learning we built a realistic simulation environment combining a Robot Operating System (ROS)-compatible physics simulator (Gazebo) with Cycles, the realistic production rendering engine from Blender. The proposed simulator pipeline allows us to simulate near-realistic RGB-D images. To showcase the capabilities of the simulator pipeline we propose a case study that focuses on indoor robotic farming. We developed a solution for sweet pepper yield estimation task. Our approach to yield estimation starts with aerial robotics control and trajectory planning, combined with deep learning-based pepper detection, and a clustering approach for counting fruit. The results of this case study show that we can combine real time dynamic simulation with near realistic rendering capabilities to simulate complex robotic systems.