Distributed quasi-Newton robust estimation under differential privacy

TL;DR

This paper introduces a robust, privacy-preserving distributed quasi-Newton estimation method that reduces communication and privacy costs while maintaining high efficiency and convergence guarantees.

Contribution

It develops a novel distributed quasi-Newton algorithm that requires minimal communication, does not depend on gradient bounds, and ensures privacy with high probability under sub-exponential distributions.

Findings

Achieves high asymptotic efficiency.

Reduces transmission rounds compared to gradient descent.

Ensures privacy with high probability under certain distribution assumptions.

Abstract

For distributed computing with Byzantine machines under Privacy Protection (PP) constraints, this paper develops a robust PP distributed quasi-Newton estimation, which only requires the node machines to transmit five vectors to the central processor with high asymptotic relative efficiency. Compared with the gradient descent strategy which requires more rounds of transmission and the Newton iteration strategy which requires the entire Hessian matrix to be transmitted, the novel quasi-Newton iteration has advantages in reducing privacy budgeting and transmission cost. Moreover, our PP algorithm does not depend on the boundedness of gradients and second-order derivatives. When gradients and second-order derivatives follow sub-exponential distributions, we offer a mechanism that can ensure PP with a sufficiently high probability. Furthermore, this novel estimator can achieve the optimal convergence rate and the asymptotic normality. The numerical studies on synthetic and real data sets evaluate the performance of the proposed algorithm.