Intrinsic Complexity And Scaling Laws: From Random Fields to Random Vectors

TL;DR

This paper characterizes the intrinsic complexity of random fields using second order statistics and derives scaling laws for this complexity as correlation length diminishes, linking continuous fields to discrete random vectors.

Contribution

It introduces a new framework for understanding the complexity of random fields through their covariance structure and establishes precise scaling laws in various settings.

Findings

Scaling laws for intrinsic complexity as correlation length approaches zero.

Connection between random fields and embeddings of random vectors.

Precise characterization for i.i.d. and structured covariance random vectors.

Abstract

Random fields are commonly used for modeling of spatially (or timely) dependent stochastic processes. In this study, we provide a characterization of the intrinsic complexity of a random field in terms of its second order statistics, e.g., the covariance function, based on the Karhumen-Lo\'{e}ve expansion. We then show scaling laws for the intrinsic complexity of a random field in terms of the correlation length as it goes to 0. In the discrete setting, it becomes approximate embeddings of a set of random vectors. We provide a precise scaling law when the random vectors have independent and identically distributed entires using random matrix theory as well as when the random vectors has a specific covariance structure.