User-Level Differential Privacy against Attribute Inference Attack of Speech Emotion Recognition in Federated Learning

TL;DR

This paper evaluates user-level differential privacy in federated learning for speech emotion recognition, demonstrating its effectiveness in reducing attribute inference attacks while maintaining system utility.

Contribution

The study applies user-level differential privacy to federated speech emotion recognition, providing theoretical privacy guarantees and analyzing its impact on privacy and utility.

Findings

UDP reduces attribute inference leakage

Utility is maintained with limited model updates leaked

Effectiveness decreases with more leaked updates

Abstract

Many existing privacy-enhanced speech emotion recognition (SER) frameworks focus on perturbing the original speech data through adversarial training within a centralized machine learning setup. However, this privacy protection scheme can fail since the adversary can still access the perturbed data. In recent years, distributed learning algorithms, especially federated learning (FL), have gained popularity to protect privacy in machine learning applications. While FL provides good intuition to safeguard privacy by keeping the data on local devices, prior work has shown that privacy attacks, such as attribute inference attacks, are achievable for SER systems trained using FL. In this work, we propose to evaluate the user-level differential privacy (UDP) in mitigating the privacy leaks of the SER system in FL. UDP provides theoretical privacy guarantees with privacy parameters $\epsilon$ and $\delta$. Our results show that the UDP can effectively decrease attribute information leakage while keeping the utility of the SER system with the adversary accessing one model update. However, the efficacy of the UDP suffers when the FL system leaks more model updates to the adversary. We make the code publicly available to reproduce the results in https://github.com/usc-sail/fed-ser-leakage.