Plant Doctor: A hybrid machine learning and image segmentation software to quantify plant damage in video footage

TL;DR

Plant Doctor is a hybrid AI system combining object detection, tracking, and image segmentation to automatically diagnose plant damage from urban street video footage, aiding in urban plant health monitoring.

Contribution

The paper introduces a novel hybrid AI system integrating YOLOv8, DeepSORT, and DeepLabV3Plus for automated plant health diagnosis from video footage, which is scalable and effective in urban environments.

Findings

High accuracy in detecting and tracking leaves in urban footage

Effective segmentation of leaf damage caused by pests and diseases

Robust performance validated on diverse urban plant datasets

Abstract

Artificial intelligence has significantly advanced the automation of diagnostic processes, benefiting various fields including agriculture. This study introduces an AI-based system for the automatic diagnosis of urban street plants using video footage obtained with accessible camera devices. The system aims to monitor plant health on a day-to-day basis, aiding in the control of disease spreading in urban areas. By combining two machine vision algorithms, YOLOv8 and DeepSORT, the system efficiently identifies and tracks individual leaves, extracting the optimal images for health analysis. YOLOv8, chosen for its speed and computational efficiency, locates leaves, while DeepSORT ensures robust tracking in complex environments. For detailed health assessment, DeepLabV3Plus, a convolutional neural network, is employed to segment and quantify leaf damage caused by bacteria, pests, and fungi. The hybrid system, named Plant Doctor, has been trained and validated using a diverse dataset including footage from Tokyo urban plants. The results demonstrate the robustness and accuracy of the system in diagnosing leaf damage, with potential applications in large scale urban flora illness monitoring. This approach provides a non-invasive, efficient, and scalable solution for urban tree health management, supporting sustainable urban ecosystems.