A framework for scale-sensitive, spatially explicit accuracy assessment of binary built-up surface layers

TL;DR

This paper develops a framework for assessing the spatially explicit accuracy of binary built-up land datasets, addressing the challenges posed by class imbalance and density variations across rural-urban areas over time.

Contribution

It introduces a novel approach to localized accuracy assessment for built-up land data, evaluating agreement measures and their sensitivity to assessment support and data structure.

Findings

Localized accuracy varies significantly with agreement measure choice.

Accuracy of built-up data generally improves over time, especially in peri-urban areas.

Densification measures may overestimate urban growth due to data limitations.

Abstract

To better understand the dynamics of human settlements, thorough knowledge of the uncertainty in geospatial built-up surface datasets is critical. While frameworks for localized accuracy assessments of categorical gridded data have been proposed to account for the spatial non-stationarity of classification accuracy, such approaches have not been applied to (binary) built-up land data. Such data differs from other data such as land cover data, due to considerable variations of built-up surface density across the rural-urban continuum resulting in switches of class imbalance, causing sparsely populated confusion matrices based on small underlying sample sizes. In this paper, we aim to fill this gap by testing common agreement measures for their suitability and plausibility to measure the localized accuracy of built-up surface data. We examine the sensitivity of localized accuracy to the assessment support, as well as to the unit of analysis, and analyze the relationships between local accuracy and density / structure-related properties of built-up areas, across rural-urban trajectories and over time. Our experiments are based on the multi-temporal Global Human Settlement Layer (GHSL) and a reference database for the state of Massachusetts (USA). We find strong variation of suitability among commonly used agreement measures, and varying levels of sensitivity to the assessment support. We then apply our framework to assess localized GHSL data accuracy over time from 1975 to 2014. Besides increasing accuracy along the rural-urban gradient, we find that accuracy generally increases over time, mainly driven by peri-urban densification processes in our study area. Moreover, we find that localized densification measures derived from the GHSL tend to overestimate peri-urban densification processes that occurred between 1975 and 2014, due to higher levels of omission errors in the GHSL epoch 1975.