Optimal Integrative Estimation for Distributed Precision Matrices with Heterogeneity Adjustment

TL;DR

This paper introduces HEAT and IteHEAT, innovative algorithms for distributed estimation of heterogeneous precision matrices, achieving statistical optimality and efficiency in communication and computation, validated through simulations and real data.

Contribution

The paper develops the first communication- and computation-efficient algorithms with proven statistical optimality for distributed heterogeneous precision matrix estimation.

Findings

HEAT achieves near-optimal convergence rates.

IteHEAT iteratively improves estimation with geometric convergence.

Algorithms perform well in simulations and real data applications.

Abstract

Distributed learning offers a practical solution for the integrative analysis of multi-source datasets, especially under privacy or communication constraints. However, addressing prospective distributional heterogeneity and ensuring communication efficiency pose significant challenges on distributed statistical analysis. In this article, we focus on integrative estimation of distributed heterogeneous precision matrices, a crucial task related to joint precision matrix estimation where computation-efficient algorithms and statistical optimality theories are still underdeveloped. To tackle these challenges, we introduce a novel HEterogeneity-adjusted Aggregating and Thresholding (HEAT) approach for distributed integrative estimation. HEAT is designed to be both communication- and computation-efficient, and we demonstrate its statistical optimality by establishing the convergence rates and the corresponding minimax lower bounds under various integrative losses. To enhance the optimality of HEAT, we further propose an iterative HEAT (IteHEAT) approach. By iteratively refining the higher-order errors of HEAT estimators through multi-round communications, IteHEAT achieves geometric contraction rates of convergence. Extensive simulations and real data applications validate the numerical performance of HEAT and IteHEAT methods.