Byzantine-Robust Distributed Sparse Learning Revisited

TL;DR

This paper presents a robust distributed estimation framework for high-dimensional sparse linear models that combines local regularization with robust aggregation, achieving near-optimal statistical guarantees and strong attack resilience.

Contribution

It introduces a novel framework that integrates local $ ext{l}_1$-regularized estimation with robust aggregation, applicable to multiple models, with proven non-asymptotic guarantees and efficiency.

Findings

Estimators achieve near-optimal statistical rates under mild conditions.

Framework demonstrates strong robustness against Byzantine attacks in simulations.

Applicable to pseudo-Huber regression, quantile regression, and sparse SVM.

Abstract

We revisit Byzantine robust distributed estimation for high-dimensional sparse linear models. By combining local $\ell_1$-regularized robust estimation with robust aggregation at the server, the framework applies to pseudo-Huber regression, quantile regression, and sparse SVM. We show that the resulting estimators yield non-asymptotic guarantees and attain near-optimal statistical rates under mild conditions, while remaining communication-efficient. Simulations confirm strong robustness in estimation, support recovery and classification accuracy under various Byzantine attacks.