Near optimal sample complexity for matrix and tensor normal models via geodesic convexity

TL;DR

This paper demonstrates that the maximum likelihood estimator for matrix and tensor normal models achieves nearly optimal sample complexity and error rates by leveraging geodesic convexity in the Fisher information geometry, without restrictive assumptions.

Contribution

It establishes nearly optimal sample complexity and error bounds for MLE in matrix and tensor normal models using geodesic convexity, independent of well-conditioned or sparse factors.

Findings

MLE achieves nearly optimal sample complexity.

Flip-flop algorithm converges linearly with high probability.

Strong geodesic convexity underpins the analysis.

Abstract

The matrix normal model, i.e., the family of Gaussian matrix-variate distributions whose covariance matrices are the Kronecker product of two lower dimensional factors, is frequently used to model matrix-variate data. The tensor normal model generalizes this family to Kronecker products of three or more factors. We study the estimation of the Kronecker factors of the covariance matrix in the matrix and tensor normal models. For the above models, we show that the maximum likelihood estimator (MLE) achieves nearly optimal nonasymptotic sample complexity and nearly tight error rates in the Fisher-Rao and Thompson metrics. In contrast to prior work, our results do not rely on the factors being well-conditioned or sparse, nor do we need to assume an accurate enough initial guess. For the matrix normal model, all our bounds are minimax optimal up to logarithmic factors, and for the tensor normal model our bounds for the largest factor and for overall covariance matrix are minimax optimal up to constant factors provided there are enough samples for any estimator to obtain constant Frobenius error. In the same regimes as our sample complexity bounds, we show that the flip-flop algorithm, a practical and widely used iterative procedure to compute the MLE, converges linearly with high probability. Our main technical insight is that, given enough samples, the negative log-likelihood function is strongly geodesically convex in the geometry on positive-definite matrices induced by the Fisher information metric. This strong convexity is determined by the expansion of certain random quantum channels.