Real-time Semantic Segmentation of Crop and Weed for Precision Agriculture Robots Leveraging Background Knowledge in CNNs

TL;DR

This paper presents a CNN-based system for real-time semantic segmentation of crops and weeds in precision agriculture, leveraging background knowledge to improve accuracy and adaptability across different fields.

Contribution

The authors introduce a CNN that incorporates vegetation indexes for real-time crop and weed segmentation, with effective re-training capabilities for unseen fields using limited data.

Findings

Operates at around 20Hz for real-time application

Generalizes well across different fields in Germany and Switzerland

Requires only a small amount of data for re-training

Abstract

Precision farming robots, which target to reduce the amount of herbicides that need to be brought out in the fields, must have the ability to identify crops and weeds in real time to trigger weeding actions. In this paper, we address the problem of CNN-based semantic segmentation of crop fields separating sugar beet plants, weeds, and background solely based on RGB data. We propose a CNN that exploits existing vegetation indexes and provides a classification in real time. Furthermore, it can be effectively re-trained to so far unseen fields with a comparably small amount of training data. We implemented and thoroughly evaluated our system on a real agricultural robot operating in different fields in Germany and Switzerland. The results show that our system generalizes well, can operate at around 20Hz, and is suitable for online operation in the fields.