weedNet: Dense Semantic Weed Classification Using Multispectral Images and MAV for Smart Farming

TL;DR

This paper presents a dense semantic weed classification method using multispectral images from a micro aerial vehicle, employing a CNN-based approach to improve accuracy in autonomous weed detection for smart farming.

Contribution

It introduces a novel application of the Segnet CNN architecture for multispectral weed classification with ground truth generation using NDVI, and demonstrates deployment on embedded systems.

Findings

Achieved approximately 0.8 F1-score and 0.78 AUC in weed classification

Developed a dataset with ground truth for weed and crop using NDVI

Successfully integrated the model with a Jetson TX2 for MAV deployment

Abstract

Selective weed treatment is a critical step in autonomous crop management as related to crop health and yield. However, a key challenge is reliable, and accurate weed detection to minimize damage to surrounding plants. In this paper, we present an approach for dense semantic weed classification with multispectral images collected by a micro aerial vehicle (MAV). We use the recently developed encoder-decoder cascaded Convolutional Neural Network (CNN), Segnet, that infers dense semantic classes while allowing any number of input image channels and class balancing with our sugar beet and weed datasets. To obtain training datasets, we established an experimental field with varying herbicide levels resulting in field plots containing only either crop or weed, enabling us to use the Normalized Difference Vegetation Index (NDVI) as a distinguishable feature for automatic ground truth generation. We train 6 models with different numbers of input channels and condition (fine-tune) it to achieve about 0.8 F1-score and 0.78 Area Under the Curve (AUC) classification metrics. For model deployment, an embedded GPU system (Jetson TX2) is tested for MAV integration. Dataset used in this paper is released to support the community and future work.