Streaming regularization parameter selection via stochastic gradient descent

TL;DR

This paper introduces an online method for selecting regularization parameters in streaming covariance estimation using stochastic gradient descent, applicable to linear and graphical models, with proven convergence and demonstrated effectiveness.

Contribution

It presents a novel online framework for regularization parameter estimation via stochastic gradient descent in streaming covariance selection, extending to graphical models.

Findings

Effective online regularization parameter estimation demonstrated on synthetic data.

Convergence results established under mild assumptions.

Successful application to neuroimaging data.

Abstract

We propose a framework to perform streaming covariance selection. Our approach employs regularization constraints where a time-varying sparsity parameter is iteratively estimated via stochastic gradient descent. This allows for the regularization parameter to be efficiently learnt in an online manner. The proposed framework is developed for linear regression models and extended to graphical models via neighbourhood selection. Under mild assumptions, we are able to obtain convergence results in a non-stochastic setting. The capabilities of such an approach are demonstrated using both synthetic data as well as neuroimaging data.