Expected Value of Matrix Quadratic Forms with Wishart distributed Random Matrices

TL;DR

This paper derives the expected value of quadratic forms involving Wishart distributed matrices to analyze stochastic gradient methods, providing formulas and examples to compare different stochastic algorithms.

Contribution

It introduces a novel derivation of the expected quadratic form involving Wishart matrices, aiding the analysis of stochastic gradient methods.

Findings

Derived an explicit formula for E(QBQ) with Wishart matrices

Provided special case analyses for the formula

Demonstrated practical application through a numerical example

Abstract

To explore the limits of a stochastic gradient method, it may be useful to consider an example consisting of an infinite number of quadratic functions. In this context, it is appropriate to determine the expected value and the covariance matrix of the stochastic noise, i.e. the difference of the true gradient and the approximated gradient generated from a finite sample. When specifying the covariance matrix, the expected value of a quadratic form QBQ is needed, where Q is a Wishart distributed random matrix and B is an arbitrary fixed symmetric matrix. After deriving an expression for E(QBQ) and considering some special cases, a numerical example is used to show how these results can support the comparison of two stochastic methods.