Residue-based Label Protection Mechanisms in Vertical Logistic Regression

TL;DR

This paper identifies privacy vulnerabilities in vertical federated logistic regression and proposes three protection mechanisms, including differential privacy and encryption, to safeguard labels with minimal impact on model accuracy.

Contribution

It introduces a novel label inference attack in vertical FL and proposes three effective privacy-preserving mechanisms combining differential privacy and encryption techniques.

Findings

Residue variables can be exploited to infer private labels.

Additive and multiplicative noise mechanisms protect labels with slight accuracy loss.

Hybrid mechanism achieves label privacy without degrading model performance.

Abstract

Federated learning (FL) enables distributed participants to collaboratively learn a global model without revealing their private data to each other. Recently, vertical FL, where the participants hold the same set of samples but with different features, has received increased attention. This paper first presents one label inference attack method to investigate the potential privacy leakages of the vertical logistic regression model. Specifically, we discover that the attacker can utilize the residue variables, which are calculated by solving the system of linear equations constructed by local dataset and the received decrypted gradients, to infer the privately owned labels. To deal with this, we then propose three protection mechanisms, e.g., additive noise mechanism, multiplicative noise mechanism, and hybrid mechanism which leverages local differential privacy and homomorphic encryption techniques, to prevent the attack and improve the robustness of the vertical logistic regression. model. Experimental results show that both the additive noise mechanism and the multiplicative noise mechanism can achieve efficient label protection with only a slight drop in model testing accuracy, furthermore, the hybrid mechanism can achieve label protection without any testing accuracy degradation, which demonstrates the effectiveness and efficiency of our protection techniques