AgriKD: Cross-Architecture Knowledge Distillation for Efficient Leaf Disease Classification

TL;DR

AgriKD introduces a cross-architecture knowledge distillation method that effectively transfers knowledge from Vision Transformers to lightweight CNNs, enabling efficient leaf disease classification on edge devices with minimal accuracy loss.

Contribution

The paper proposes a novel multi-level distillation framework bridging Transformer and CNN architectures, significantly improving lightweight model performance for agricultural disease detection.

Findings

Student model achieves comparable accuracy to the teacher.

Model parameters reduced by approximately 172 times.

Inference latency decreased by 18-22 times.

Abstract

Automated leaf disease classification is critical for early disease detection in resource-constrained field environments. Vision Transformers (ViTs) provide strong representation capability by modeling long-range dependencies and inter-class relationships; however, their high computational cost makes them impractical for deployment on edge devices. As a result, existing approaches struggle to effectively transfer these rich representations to lightweight models. This paper introduces AgriKD, a cross-architecture knowledge distillation framework for efficient edge deployment, which transfers knowledge from a Vision Transformer (ViT) teacher to a compact convolutional student model. To bridge the representational gap between Transformer and CNN architectures, the proposed approach integrates multiple distillation objectives at the output, feature, and relational levels, where each objective captures a different aspect of the teacher knowledge. This enables the student model to better preserve and utilize transformer-derived global representations. Experiments on multiple leaf disease datasets show that the distilled student achieves performance comparable to the teacher while significantly improving efficiency, reducing model parameters by approximately 172 times, computational cost by 47.57 times, and inference latency by 18-22 times. Furthermore, the optimized model is deployed across multiple runtime formats, including ONNX, TFLite Float16, and TensorRT FP16, achieving consistent predictive performance with negligible accuracy degradation. Real-world deployment on NVIDIA Jetson edge devices and a mobile application demonstrates reliable real-time inference, highlighting the practicality of AgriKD for AI-powered agricultural applications in resource-constrained environments.