Privacy-aware Berrut Approximated Coded Computing applied to general distributed learning

TL;DR

This paper introduces Private Berrut Approximate Coded Computing (PBACC), a novel method that enhances privacy in federated learning across various models with minimal impact on learning quality and low privacy leakage.

Contribution

The paper develops new PBACC algorithms for centralized, secure distributed, and decentralized training, broadening the applicability of privacy-preserving coded computing in federated learning.

Findings

PBACC minimally affects model accuracy across CNNs, VAEs, and Cox regression.

Privacy leakage is strictly limited to less than one bit per participant.

Encoding and decoding costs depend only on data decentralization degree.

Abstract

Coded computing is one of the techniques that can be used for privacy protection in Federated Learning. However, most of the constructions used for coded computing work only under the assumption that the computations involved are exact, generally restricted to special classes of functions, and require quantized inputs. This paper considers the use of Private Berrut Approximate Coded Computing (PBACC) as a general solution to add strong but non-perfect privacy to federated learning. We derive new adapted PBACC algorithms for centralized aggregation, secure distributed training with centralized data, and secure decentralized training with decentralized data, thus enlarging significantly the applications of the method and the existing privacy protection tools available for these paradigms. Particularly, PBACC can be used robustly to attain privacy guarantees in decentralized federated learning for a variety of models. Our numerical results show that the achievable quality of different learning models (convolutional neural networks, variational autoencoders, and Cox regression) is minimally altered by using these new computing schemes, and that the privacy leakage can be bounded strictly to less than a fraction of one bit per participant. Additionally, the computational cost of the encoding and decoding processes depends only of the degree of decentralization of the data.