Auxiliary-task learning for geographic data with autoregressive embeddings

TL;DR

This paper introduces SXL, a novel auxiliary-task learning method that embeds autoregressive spatial information into neural networks using multi-resolution Moran's I, improving spatial data modeling and outperforming benchmarks.

Contribution

The study presents a new multi-resolution Moran's I expansion and integrates it into neural networks as an auxiliary task, enhancing spatial data learning.

Findings

Consistent improvement in neural network training for spatial data tasks.

Outperforms domain-specific spatial interpolation benchmarks.

Effective in both supervised and unsupervised learning scenarios.

Abstract

Machine learning is gaining popularity in a broad range of areas working with geographic data, such as ecology or atmospheric sciences. Here, data often exhibit spatial effects, which can be difficult to learn for neural networks. In this study, we propose SXL, a method for embedding information on the autoregressive nature of spatial data directly into the learning process using auxiliary tasks. We utilize the local Moran's I, a popular measure of local spatial autocorrelation, to "nudge" the model to learn the direction and magnitude of local spatial effects, complementing the learning of the primary task. We further introduce a novel expansion of Moran's I to multiple resolutions, thus capturing spatial interactions over longer and shorter distances simultaneously. The novel multi-resolution Moran's I can be constructed easily and as a multi-dimensional tensor offers seamless integration into existing machine learning frameworks. Throughout a range of experiments using real-world data, we highlight how our method consistently improves the training of neural networks in unsupervised and supervised learning tasks. In generative spatial modeling experiments, we propose a novel loss for auxiliary task GANs utilizing task uncertainty weights. Our proposed method outperforms domain-specific spatial interpolation benchmarks, highlighting its potential for downstream applications. This study bridges expertise from geographic information science and machine learning, showing how this integration of disciplines can help to address domain-specific challenges. The code for our experiments is available on Github: https://github.com/konstantinklemmer/sxl.