Estimation of Constrained Mean-Covariance of Normal Distributions

TL;DR

This paper introduces a new method for estimating the mean and covariance of multivariate normal distributions under a specific linear constraint, improving efficiency and addressing positive-definiteness issues.

Contribution

It proposes a novel structured covariance parameterization and a fast, noniterative approximation method for maximum likelihood estimation under the constraint.

Findings

The method effectively estimates parameters in simulations.

The structured covariance reduces parameters from quadratic to linear.

The approach performs well compared to traditional methods.

Abstract

Estimation of the mean vector and covariance matrix is of central importance in the analysis of multivariate data. In the framework of generalized linear models, usually the variances are certain functions of the means with the normal distribution being an exception. We study some implications of functional relationships between covariance and the mean by focusing on the maximum likelihood and Bayesian estimation of the mean-covariance under the joint constraint $\bm{\Sigma}\bm{\mu} = \bm{\mu}$ for a multivariate normal distribution. A novel structured covariance is proposed through reparameterization of the spectral decomposition of $\bm{\Sigma}$ involving its eigenvalues and $\bm{\mu}$. This is designed to address the challenging issue of positive-definiteness and to reduce the number of covariance parameters from quadratic to linear function of the dimension. We propose a fast (noniterative) method for approximating the maximum likelihood estimator by maximizing a lower bound for the profile likelihood function, which is concave. We use normal and inverse gamma priors on the mean and eigenvalues, and approximate the maximum aposteriori estimators by both MH within Gibbs sampling and a faster iterative method. A simulation study shows good performance of our estimators.