A Fast and Scalable Joint Estimator for Learning Multiple Related Sparse Gaussian Graphical Models

TL;DR

This paper introduces FASJEM, a fast, scalable method for jointly estimating multiple sparse Gaussian Graphical Models, significantly improving efficiency and accuracy in high-dimensional, multi-task settings.

Contribution

First joint sGGM estimator based on the Elementary Estimator framework, with parallelizable entry-wise optimization and proven consistency with improved computational and memory efficiency.

Findings

FASJEM reduces computational complexity from O(Kp^3) to O(Kp^2).

FASJEM achieves consistent estimation with a convergence rate of O(log(Kp)/n_{tot}).

FASJEM outperforms baselines in accuracy, speed, and memory on synthetic and real datasets.

Abstract

Estimating multiple sparse Gaussian Graphical Models (sGGMs) jointly for many related tasks (large $K$) under a high-dimensional (large $p$) situation is an important task. Most previous studies for the joint estimation of multiple sGGMs rely on penalized log-likelihood estimators that involve expensive and difficult non-smooth optimizations. We propose a novel approach, FASJEM for \underline{fa}st and \underline{s}calable \underline{j}oint structure-\underline{e}stimation of \underline{m}ultiple sGGMs at a large scale. As the first study of joint sGGM using the Elementary Estimator framework, our work has three major contributions: (1) We solve FASJEM through an entry-wise manner which is parallelizable. (2) We choose a proximal algorithm to optimize FASJEM. This improves the computational efficiency from $O(Kp^3)$ to $O(Kp^2)$ and reduces the memory requirement from $O(Kp^2)$ to $O(K)$. (3) We theoretically prove that FASJEM achieves a consistent estimation with a convergence rate of $O(\log(Kp)/n_{tot})$. On several synthetic and four real-world datasets, FASJEM shows significant improvements over baselines on accuracy, computational complexity, and memory costs.