Optical flow-based branch segmentation for complex orchard environments

TL;DR

This paper presents a neural network trained solely on simulated RGB and optical flow data to accurately segment orchard branches, demonstrating robustness and consistency across diverse real-world environments without additional real-world training.

Contribution

The novel approach trains a neural network in simulation using only RGB and optical flow data, eliminating the need for real-world labeled datasets or specialized equipment.

Findings

Achieves high accuracy in branch segmentation in complex orchard environments.

Outperforms networks trained on manually labeled RGBD data in robustness and consistency.

Operates effectively without additional real-world training or specialized hardware.

Abstract

Machine vision is a critical subsystem for enabling robots to be able to perform a variety of tasks in orchard environments. However, orchards are highly visually complex environments, and computer vision algorithms operating in them must be able to contend with variable lighting conditions and background noise. Past work on enabling deep learning algorithms to operate in these environments has typically required large amounts of hand-labeled data to train a deep neural network or physically controlling the conditions under which the environment is perceived. In this paper, we train a neural network system in simulation only using simulated RGB data and optical flow. This resulting neural network is able to perform foreground segmentation of branches in a busy orchard environment without additional real-world training or using any special setup or equipment beyond a standard camera. Our results show that our system is highly accurate and, when compared to a network using manually labeled RGBD data, achieves significantly more consistent and robust performance across environments that differ from the training set.