Rethinking Plant Disease Diagnosis: Bridging the Academic-Practical Gap with Vision Transformers and Zero-Shot Learning

TL;DR

This paper explores how vision transformers and zero-shot learning can improve plant disease diagnosis in real-world conditions, addressing the limitations of traditional models trained on curated datasets.

Contribution

It demonstrates that vision transformers and CLIP-based zero-shot models outperform CNNs in generalizing to field images, offering scalable solutions for practical plant disease diagnosis.

Findings

Vision transformers show better robustness than CNNs under domain shift.

CLIP models classify diseases without task-specific training, enhancing adaptability.

Zero-shot learning offers a scalable approach for real-world plant health diagnosis.

Abstract

Recent advances in deep learning have enabled significant progress in plant disease classification using leaf images. Much of the existing research in this field has relied on the PlantVillage dataset, which consists of well-centered plant images captured against uniform, uncluttered backgrounds. Although models trained on this dataset achieve high accuracy, they often fail to generalize to real-world field images, such as those submitted by farmers to plant diagnostic systems. This has created a significant gap between published studies and practical application requirements, highlighting the necessity of investigating and addressing this issue. In this study, we investigate whether attention-based architectures and zero-shot learning approaches can bridge the gap between curated academic datasets and real-world agricultural conditions in plant disease classification. We evaluate three model categories: Convolutional Neural Networks (CNNs), Vision Transformers, and Contrastive Language-Image Pre-training (CLIP)-based zero-shot models. While CNNs exhibit limited robustness under domain shift, Vision Transformers demonstrate stronger generalization by capturing global contextual features. Most notably, CLIP models classify diseases directly from natural language descriptions without any task-specific training, offering strong adaptability and interpretability. These findings highlight the potential of zero-shot learning as a practical and scalable domain adaptation strategy for plant health diagnosis in diverse field environments.