Resilient Peer-to-peer Learning based on Adaptive Aggregation

TL;DR

This paper proposes a resilient peer-to-peer learning method with adaptive aggregation that enhances robustness against adversarial attacks, preserves data privacy, and ensures convergence in complex, real-world scenarios.

Contribution

It introduces a novel aggregation technique optimized for non-convex, non-iid data, with theoretical convergence guarantees and empirical validation against diverse attack models.

Findings

Improved accuracy over existing methods.

Robustness against various adversarial attacks.

Convergence demonstrated for non-convex, non-iid data.

Abstract

Collaborative learning in peer-to-peer networks offers the benefits of distributed learning while mitigating the risks associated with single points of failure inherent in centralized servers. However, adversarial workers pose potential threats by attempting to inject malicious information into the network. Thus, ensuring the resilience of peer-to-peer learning emerges as a pivotal research objective. The challenge is exacerbated in the presence of non-convex loss functions and non-iid data distributions. This paper introduces a resilient aggregation technique tailored for such scenarios, aimed at fostering similarity among peers' learning processes. The aggregation weights are determined through an optimization procedure, and use the loss function computed using the neighbor's models and individual private data, thereby addressing concerns regarding data privacy in distributed machine learning. Theoretical analysis demonstrates convergence of parameters with non-convex loss functions and non-iid data distributions. Empirical evaluations across three distinct machine learning tasks support the claims. The empirical findings, which encompass a range of diverse attack models, also demonstrate improved accuracy when compared to existing methodologies.