AI applications in forest monitoring need remote sensing benchmark datasets

TL;DR

This paper highlights the need for standardized benchmarking datasets in remote sensing-based forest monitoring to improve method comparison, accuracy, and progress in the field.

Contribution

It discusses the impact of data heterogeneity on analysis performance and proposes pragmatic requirements for creating effective benchmarking datasets.

Findings

Lack of standardization affects confidence in forest property estimation.

Benchmarking datasets are essential for method comparison and validation.

Community-driven initiatives can enhance dataset development and field progress.

Abstract

With the rise in high resolution remote sensing technologies there has been an explosion in the amount of data available for forest monitoring, and an accompanying growth in artificial intelligence applications to automatically derive forest properties of interest from these datasets. Many studies use their own data at small spatio-temporal scales, and demonstrate an application of an existing or adapted data science method for a particular task. This approach often involves intensive and time-consuming data collection and processing, but generates results restricted to specific ecosystems and sensor types. There is a lack of widespread acknowledgement of how the types and structures of data used affects performance and accuracy of analysis algorithms. To accelerate progress in the field more efficiently, benchmarking datasets upon which methods can be tested and compared are sorely needed. Here, we discuss how lack of standardisation impacts confidence in estimation of key forest properties, and how considerations of data collection need to be accounted for in assessing method performance. We present pragmatic requirements and considerations for the creation of rigorous, useful benchmarking datasets for forest monitoring applications, and discuss how tools from modern data science can improve use of existing data. We list a set of example large-scale datasets that could contribute to benchmarking, and present a vision for how community-driven, representative benchmarking initiatives could benefit the field.