Probing the Information Theoretical Roots of Spatial Dependence Measures

TL;DR

This paper explores the theoretical connection between spatial dependence measures and information theory, specifically analyzing Moran's I through the concept of self-information, supported by formal proofs and experiments.

Contribution

It formalizes the relationship between spatial autocorrelation and information theory, providing new theoretical insights and experimental validation.

Findings

Spatial data contains less information than expected, explaining its autocorrelation.

Spatially autocorrelated data is more compressible, linking compression and spatial dependence.

Formal proofs establish the theoretical basis for the relation between Moran's I and information measures.

Abstract

Intuitively, there is a relation between measures of spatial dependence and information theoretical measures of entropy. For instance, we can provide an intuition of why spatial data is special by stating that, on average, spatial data samples contain less than expected information. Similarly, spatial data, e.g., remotely sensed imagery, that is easy to compress is also likely to show significant spatial autocorrelation. Formulating our (highly specific) core concepts of spatial information theory in the widely used language of information theory opens new perspectives on their differences and similarities and also fosters cross-disciplinary collaboration, e.g., with the broader AI/ML communities. Interestingly, however, this intuitive relation is challenging to formalize and generalize, leading prior work to rely mostly on experimental results, e.g., for describing landscape patterns. In this work, we will explore the information theoretical roots of spatial autocorrelation, more specifically Moran's I, through the lens of self-information (also known as surprisal) and provide both formal proofs and experiments.