MSFNet-CPD: Multi-Scale Cross-Modal Fusion Network for Crop Pest Detection

TL;DR

This paper introduces MSFNet-CPD, a multi-modal deep learning framework that enhances crop pest detection accuracy by integrating visual and textual features, utilizing super-resolution, and creating diverse datasets for better generalization.

Contribution

The paper presents a novel multi-scale cross-modal fusion network with modules for super-resolution, image-text fusion, and dataset augmentation, advancing pest detection capabilities.

Findings

MSFNet-CPD outperforms existing methods on multiple benchmarks.

The super-resolution module improves image clarity and detection accuracy.

The dataset augmentation strategy enhances model generalization.

Abstract

Accurate identification of agricultural pests is essential for crop protection but remains challenging due to the large intra-class variance and fine-grained differences among pest species. While deep learning has advanced pest detection, most existing approaches rely solely on low-level visual features and lack effective multi-modal integration, leading to limited accuracy and poor interpretability. Moreover, the scarcity of high-quality multi-modal agricultural datasets further restricts progress in this field. To address these issues, we construct two novel multi-modal benchmarks-CTIP102 and STIP102-based on the widely-used IP102 dataset, and introduce a Multi-scale Cross-Modal Fusion Network (MSFNet-CPD) for robust pest detection. Our approach enhances visual quality via a super-resolution reconstruction module, and feeds both the original and reconstructed images into the network to improve clarity and detection performance. To better exploit semantic cues, we propose an Image-Text Fusion (ITF) module for joint modeling of visual and textual features, and an Image-Text Converter (ITC) that reconstructs fine-grained details across multiple scales to handle challenging backgrounds. Furthermore, we introduce an Arbitrary Combination Image Enhancement (ACIE) strategy to generate a more complex and diverse pest detection dataset, MTIP102, improving the model's generalization to real-world scenarios. Extensive experiments demonstrate that MSFNet-CPD consistently outperforms state-of-the-art methods on multiple pest detection benchmarks. All code and datasets will be made publicly available at: https://github.com/Healer-ML/MSFNet-CPD.