Towards Robust Deep Learning-based Rumex Obtusifolius Detection from Drone Images

TL;DR

This study compares domain adaptation techniques and pretrained Vision Transformers for robust Rumex obtusifolius weed detection from drone images, highlighting ViTs' superior intrinsic domain generalization.

Contribution

It demonstrates that pretrained Vision Transformers outperform CNNs with domain adaptation methods in UAV-based weed classification tasks.

Findings

ViTs pretrained with self-supervised learning handle domain shifts well.

Domain adaptation improves CNN performance but is surpassed by ViTs.

High classification accuracy (F1=0.8) achieved with ViTs fine-tuned on source data.

Abstract

Domain adaptation (DA) addresses the challenge of transferring a machine learning model trained on a source domain to a target domain with a different data distribution. In this work, we study DA for the task of Rumex obtusifolius (Rumex) image classification. We train models on a published, ground vehicle-based dataset (source) and evaluate their performance on a custom target dataset acquired by unmanned aerial vehicles (UAVs). We find that Convolutional Neural Network (CNN) models, specifically ResNets, generalize poorly to the target domain, even after fine-tuning on the source data. Applying moment-matching and maximum classifier discrepancy, two established DA techniques, substantially improves target-domain performance. However, Vision Transformer (ViT) models pretrained with self-supervised objectives (DINOv2, DINOv3) handle domain shifts intrinsically well, surpassing even moment-matching-trained ResNets, likely due to the rich, general-purpose representations acquired during large-scale pretraining. Using ViTs fine-tuned on the source dataset, we demonstrate high classification performances in the range of F1=0.8 on our target dataset. To support further research on DA for weed detection in grassland systems, we publicly release our UAV-based target dataset AGSMultiRumex, comprising data from 15 flights over Swiss meadows.