Deep Random Features for Scalable Interpolation of Spatiotemporal Data

TL;DR

This paper introduces a scalable deep learning approach using random features to interpolate complex spatiotemporal data from remote sensing, overcoming Gaussian process limitations.

Contribution

It proposes a Bayesian deep learning method with random feature expansions to efficiently model non-stationary, high-frequency patterns in large-scale spatiotemporal data.

Findings

Achieves competitive or superior interpolation accuracy.

Produces well-calibrated uncertainty estimates.

Handles large-scale remote sensing datasets effectively.

Abstract

The rapid growth of earth observation systems calls for a scalable approach to interpolate remote-sensing observations. These methods in principle, should acquire more information about the observed field as data grows. Gaussian processes (GPs) are candidate model choices for interpolation. However, due to their poor scalability, they usually rely on inducing points for inference, which restricts their expressivity. Moreover, commonly imposed assumptions such as stationarity prevents them from capturing complex patterns in the data. While deep GPs can overcome this issue, training and making inference with them are difficult, again requiring crude approximations via inducing points. In this work, we instead approach the problem through Bayesian deep learning, where spatiotemporal fields are represented by deep neural networks, whose layers share the inductive bias of stationary GPs on the plane/sphere via random feature expansions. This allows one to (1) capture high frequency patterns in the data, and (2) use mini-batched gradient descent for large scale training. We experiment on various remote sensing data at local/global scales, showing that our approach produce competitive or superior results to existing methods, with well-calibrated uncertainties.