Crossmodal learning for Crop Canopy Trait Estimation

TL;DR

This paper introduces a cross-modal learning approach that enhances satellite imagery with UAV-level detail to improve crop trait estimation, enabling better agricultural monitoring and prediction tasks.

Contribution

The study presents a novel method for enriching satellite images with UAV detail through cross-modal learning, improving crop trait estimation accuracy.

Findings

Generated UAV-like representations outperform real satellite images in prediction tasks.

Cross-modal learning bridges the gap between satellite and UAV sensing.

Method improves yield and nitrogen prediction accuracy.

Abstract

Recent advances in plant phenotyping have driven widespread adoption of multi sensor platforms for collecting crop canopy reflectance data. This includes the collection of heterogeneous data across multiple platforms, with Unmanned Aerial Vehicles (UAV) seeing significant usage due to their high performance in crop monitoring, forecasting, and prediction tasks. Similarly, satellite missions have been shown to be effective for agriculturally relevant tasks. In contrast to UAVs, such missions are bound to the limitation of spatial resolution, which hinders their effectiveness for modern farming systems focused on micro-plot management. In this work, we propose a cross modal learning strategy that enriches high-resolution satellite imagery with UAV level visual detail for crop canopy trait estimation. Using a dataset of approximately co registered satellite UAV image pairs collected from replicated plots of 84 hybrid maize varieties across five distinct locations in the U.S. Corn Belt, we train a model that learns fine grained spectral spatial correspondences between sensing modalities. Results show that the generated UAV-like representations from satellite inputs consistently outperform real satellite imagery on multiple downstream tasks, including yield and nitrogen prediction, demonstrating the potential of cross-modal correspondence learning to bridge the gap between satellite and UAV sensing in agricultural monitoring.