Multi-task Learning for Gaussian Graphical Regressions with High Dimensional Covariates

TL;DR

This paper introduces a multi-task learning approach for Gaussian graphical regression that improves estimation accuracy in high-dimensional settings by leveraging shared information across tasks and employing efficient algorithms.

Contribution

It proposes a novel multi-task learning estimator with structured sparsity penalties and provides theoretical error bounds showing significant improvements over traditional methods.

Findings

Enhanced estimation accuracy demonstrated through simulations

Theoretical error bounds show substantial improvement over separate node-wise methods

Application to gene co-expression networks illustrates practical utility

Abstract

Gaussian graphical regression is a powerful means that regresses the precision matrix of a Gaussian graphical model on covariates, permitting the numbers of the response variables and covariates to far exceed the sample size. Model fitting is typically carried out via separate node-wise lasso regressions, ignoring the network-induced structure among these regressions. Consequently, the error rate is high, especially when the number of nodes is large. We propose a multi-task learning estimator for fitting Gaussian graphical regression models; we design a cross-task group sparsity penalty and a within task element-wise sparsity penalty, which govern the sparsity of active covariates and their effects on the graph, respectively. For computation, we consider an efficient augmented Lagrangian algorithm, which solves subproblems with a semi-smooth Newton method. For theory, we show that the error rate of the multi-task learning based estimates has much improvement over that of the separate node-wise lasso estimates, because the cross-task penalty borrows information across tasks. To address the main challenge that the tasks are entangled in a complicated correlation structure, we establish a new tail probability bound for correlated heavy-tailed (sub-exponential) variables with an arbitrary correlation structure, a useful theoretical result in its own right. Finally, the utility of our method is demonstrated through simulations as well as an application to a gene co-expression network study with brain cancer patients.