The Goodness of Covariance Selection Problem from AUC Bounds

TL;DR

This paper analyzes the effectiveness of covariance selection models for Gaussian vectors using AUC bounds, showing that higher model order improves selection quality and is asymptotically reliable.

Contribution

It introduces the correlation approximation matrix (CAM) and provides theoretical analysis of model selection quality for Gaussian graphical models using AUC bounds.

Findings

Model selection quality improves with increasing model order.

Asymptotic goodness of model selection when model order is proportional to number of nodes.

Simulations confirm theoretical predictions about model quality and AUC bounds.

Abstract

We conduct a study of graphical models and discuss the quality of model selection approximation by formulating the problem as a detection problem and examining the area under the curve (AUC). We are specifically looking at the model selection problem for jointly Gaussian random vectors. For Gaussian random vectors, this problem simplifies to the covariance selection problem which is widely discussed in literature by Dempster [1]. In this paper, we give the definition for the correlation approximation matrix (CAM) which contains all information about the model selection problem and discuss the pth order Markov chain model and the $p$th order star network model for the a Gaussian distribution with Toeplitz covariance matrix. For each model, we compute the model covariance matrix as well as the KL divergence between the Gaussian distribution and its model. We also show that if the model order, p, is proportional to the number of nodes, n, then the model selection is asymptotically good as the number of nodes, n, goes to infinity since the AUC in this case is bounded away from one. We conduct some simulations which confirm the theoretical analysis and also show that the selected model quality increases as the model order, p, increases.