On the Kullback-Leibler divergence between pairwise isotropic Gaussian-Markov random fields

TL;DR

This paper derives explicit formulas for the Kullback-Leibler divergence between pairwise isotropic Gaussian-Markov random fields, enabling new similarity measures for applications like image denoising and metric learning.

Contribution

It provides the first closed-form expressions for KL divergence between these Gaussian-Markov random fields in univariate and multivariate cases.

Findings

Closed-form KL divergence formulas derived

Facilitates new similarity measures in image processing

Applicable to machine learning tasks like denoising

Abstract

The Kullback-Leibler divergence or relative entropy is an information-theoretic measure between statistical models that play an important role in measuring a distance between random variables. In the study of complex systems, random fields are mathematical structures that models the interaction between these variables by means of an inverse temperature parameter, responsible for controlling the spatial dependence structure along the field. In this paper, we derive closed-form expressions for the Kullback-Leibler divergence between two pairwise isotropic Gaussian-Markov random fields in both univariate and multivariate cases. The proposed equation allows the development of novel similarity measures in image processing and machine learning applications, such as image denoising and unsupervised metric learning.