Efficient Distributed Estimation of Inverse Covariance Matrices

TL;DR

This paper introduces a communication-efficient distributed method for estimating sparse inverse covariance matrices, enabling accurate estimation with minimal data transfer across machines, and maintaining model selection accuracy.

Contribution

It proposes a novel single-round communication approach that allows distributed estimation of inverse covariance matrices with error rates comparable to centralized methods.

Findings

Error rates similar to non-distributed estimation.

Successful model selection in distributed setting.

Effective in practical simulations.

Abstract

In distributed systems, communication is a major concern due to issues such as its vulnerability or efficiency. In this paper, we are interested in estimating sparse inverse covariance matrices when samples are distributed into different machines. We address communication efficiency by proposing a method where, in a single round of communication, each machine transfers a small subset of the entries of the inverse covariance matrix. We show that, with this efficient distributed method, the error rates can be comparable with estimation in a non-distributed setting, and correct model selection is still possible. Practical performance is shown through simulations.