The augmented NLP bound for maximum-entropy remote sampling

TL;DR

This paper introduces an augmented NLP bound and a diagonal-scaling technique for the maximum-entropy remote sampling problem, improving upper bounds and extending applicability to rank-deficient covariance matrices.

Contribution

It establishes domination results between existing bounds, proposes a new augmented NLP bound with theoretical guarantees, and introduces a diagonal-scaling method for better upper bounds.

Findings

The augmented NLP bound strictly dominates the NLP bound under certain conditions.

The new bounds are effective for rank-deficient covariance matrices.

Numerical experiments show improved upper bounds on benchmark instances.

Abstract

The maximum-entropy remote sampling problem (MERSP) is to select a subset of s random variables from a set of n random variables, so as to maximize the information concerning a set of target random variables that are not directly observable. We assume throughout that the set of all of these random variables follows a joint Gaussian distribution, and that we have the covariance matrix available. Finally, we measure information using Shannon's differential entropy. The main approach for exact solution of moderate-sized instances of MERSP has been branch-and-bound, and so previous work concentrated on upper bounds. Prior to our work, there were two upper-bounding methods for MERSP: the so-called NLP bound and the spectral bound, both introduced 25 years ago. We are able now to establish domination results between these two upper bounds. We propose an ``augmented NLP bound'' based on a subtle convex relaxation. We provide theoretical guarantees, giving sufficient conditions under which the augmented NLP bound strictly dominates the ordinary NLP bound. In addition, the augmented NLP formulation allows us to derive upper bounds for rank-deficient covariance matrices when they satisfy a technical condition. This is in contrast to the earlier work on the ordinary NLP bound that worked with only positive definite covariance matrices. Finally, we introduce a novel and very effective diagonal-scaling technique for MERSP, employing a positive vector of parameters. Numerical experiments on benchmark instances demonstrate the effectiveness of our approaches in advancing the state of the art for calculating upper bounds on MERSP.