Maximizing Uncertainty for Federated learning via Bayesian Optimisation-based Model Poisoning

TL;DR

This paper introduces Delphi, a novel model poisoning attack in federated learning that maximizes model output uncertainty using Bayesian Optimization, revealing vulnerabilities in trustworthiness and privacy.

Contribution

The paper proposes Delphi, a new attack method employing Bayesian Optimization to increase uncertainty in federated learning models, demonstrating a novel security threat.

Findings

Delphi-BO induces higher uncertainty than Delphi-LSTR.

The attack effectively compromises model trustworthiness.

Mathematical proof confirms attack effectiveness in FL.

Abstract

As we transition from Narrow Artificial Intelligence towards Artificial Super Intelligence, users are increasingly concerned about their privacy and the trustworthiness of machine learning (ML) technology. A common denominator for the metrics of trustworthiness is the quantification of uncertainty inherent in DL algorithms, and specifically in the model parameters, input data, and model predictions. One of the common approaches to address privacy-related issues in DL is to adopt distributed learning such as federated learning (FL), where private raw data is not shared among users. Despite the privacy-preserving mechanisms in FL, it still faces challenges in trustworthiness. Specifically, the malicious users, during training, can systematically create malicious model parameters to compromise the models predictive and generative capabilities, resulting in high uncertainty about their reliability. To demonstrate malicious behaviour, we propose a novel model poisoning attack method named Delphi which aims to maximise the uncertainty of the global model output. We achieve this by taking advantage of the relationship between the uncertainty and the model parameters of the first hidden layer of the local model. Delphi employs two types of optimisation , Bayesian Optimisation and Least Squares Trust Region, to search for the optimal poisoned model parameters, named as Delphi-BO and Delphi-LSTR. We quantify the uncertainty using the KL Divergence to minimise the distance of the predictive probability distribution towards an uncertain distribution of model output. Furthermore, we establish a mathematical proof for the attack effectiveness demonstrated in FL. Numerical results demonstrate that Delphi-BO induces a higher amount of uncertainty than Delphi-LSTR highlighting vulnerability of FL systems to model poisoning attacks.