Byzantine-robust distributed one-step estimation

TL;DR

This paper introduces ROSE, a robust one-step estimator for distributed M-estimation that effectively handles Byzantine failures and anomalous data, with proven asymptotic properties and demonstrated practical robustness.

Contribution

The paper presents ROSE, a novel robust estimator combining VRMOL and Newton-Raphson methods for Byzantine-resilient distributed estimation, with proven efficiency and robustness.

Findings

ROSE achieves higher asymptotic efficiency than median-based estimators.

ROSE effectively handles Byzantine failures and anomalous data.

Numerical and real data experiments confirm robustness and effectiveness.

Abstract

This paper proposes a Robust One-Step Estimator(ROSE) to solve the Byzantine failure problem in distributed M-estimation when a moderate fraction of node machines experience Byzantine failures. To define ROSE, the algorithms use the robust Variance Reduced Median Of the Local(VRMOL) estimator to determine the initial parameter value for iteration, and communicate between the node machines and the central processor in the Newton-Raphson iteration procedure to derive the robust VRMOL estimator of the gradient, and the Hessian matrix so as to obtain the final estimator. ROSE has higher asymptotic relative efficiency than general median estimators without increasing the order of computational complexity. Moreover, this estimator can also cope with the problems involving anomalous or missing samples on the central processor. We prove the asymptotic normality when the parameter dimension p diverges as the sample size goes to infinity, and under weaker assumptions, derive the convergence rate. Numerical simulations and a real data application are conducted to evidence the effectiveness and robustness of ROSE.