Combining Deep Learning and Street View Imagery to Map Smallholder Crop Types

TL;DR

This paper presents an automated deep learning system that leverages street view imagery to generate high-resolution, wall-to-wall crop type maps in smallholder regions, significantly reducing the need for extensive ground truth data.

Contribution

It introduces a novel pipeline combining weakly-labeled street view images with remote sensing data to produce detailed crop maps, a first for smallholder countries at 10m resolution.

Findings

Achieved 93% accuracy in Thailand for multiple crop types.

Produced the first 10m-resolution crop map for an entire smallholder country.

Demonstrated scalability of the method with expanding roadside imagery sources.

Abstract

Accurate crop type maps are an essential source of information for monitoring yield progress at scale, projecting global crop production, and planning effective policies. To date, however, crop type maps remain challenging to create in low and middle-income countries due to a lack of ground truth labels for training machine learning models. Field surveys are the gold standard in terms of accuracy but require an often-prohibitively large amount of time, money, and statistical capacity. In recent years, street-level imagery, such as Google Street View, KartaView, and Mapillary, has become available around the world. Such imagery contains rich information about crop types grown at particular locations and times. In this work, we develop an automated system to generate crop type ground references using deep learning and Google Street View imagery. The method efficiently curates a set of street view images containing crop fields, trains a model to predict crop type by utilizing weakly-labelled images from disparate out-of-domain sources, and combines predicted labels with remote sensing time series to create a wall-to-wall crop type map. We show that, in Thailand, the resulting country-wide map of rice, cassava, maize, and sugarcane achieves an accuracy of 93%. We publicly release the first-ever crop type map for all of Thailand 2022 at 10m-resolution with no gaps. To our knowledge, this is the first time a 10m-resolution, multi-crop map has been created for any smallholder country. As the availability of roadside imagery expands, our pipeline provides a way to map crop types at scale around the globe, especially in underserved smallholder regions.