MYSTIKO : : Cloud-Mediated, Private, Federated Gradient Descent

TL;DR

This paper introduces MYSTIKO, a cloud-mediated federated learning method that uses homomorphic encryption to enhance privacy without sacrificing model accuracy or requiring complex hyperparameter tuning.

Contribution

It proposes a secure federated gradient descent approach using Paillier encryption that preserves accuracy, minimizes trust, and reduces overhead without adding noise.

Findings

Achieves privacy without accuracy loss

Eliminates need for hyperparameter tuning

Maintains efficiency and low overhead

Abstract

Federated learning enables multiple, distributed participants (potentially on different clouds) to collaborate and train machine/deep learning models by sharing parameters/gradients. However, sharing gradients, instead of centralizing data, may not be as private as one would expect. Reverse engineering attacks on plaintext gradients have been demonstrated to be practically feasible. Existing solutions for differentially private federated learning, while promising, lead to less accurate models and require nontrivial hyperparameter tuning. In this paper, we examine the use of additive homomorphic encryption (specifically the Paillier cipher) to design secure federated gradient descent techniques that (i) do not require addition of statistical noise or hyperparameter tuning, (ii) does not alter the final accuracy or utility of the final model, (iii) ensure that the plaintext model parameters/gradients of a participant are never revealed to any other participant or third party coordinator involved in the federated learning job, (iv) minimize the trust placed in any third party coordinator and (v) are efficient, with minimal overhead, and cost effective.