Investigating Effective Speaker Property Privacy Protection in Federated Learning for Speech Emotion Recognition

TL;DR

This paper investigates privacy vulnerabilities in federated learning for speech emotion recognition and proposes a novel property decomposition method with perturbations to enhance privacy while maintaining model utility.

Contribution

It introduces a new property-based perturbation approach to protect speech data properties in federated learning for SER, improving privacy-utility balance.

Findings

Proposed method outperforms existing privacy protection techniques.

Maintains model utility while significantly reducing inference attack risks.

Demonstrates effective privacy-utility trade-offs in experiments.

Abstract

Federated Learning (FL) is a privacy-preserving approach that allows servers to aggregate distributed models transmitted from local clients rather than training on user data. More recently, FL has been applied to Speech Emotion Recognition (SER) for secure human-computer interaction applications. Recent research has found that FL is still vulnerable to inference attacks. To this end, this paper focuses on investigating the security of FL for SER concerning property inference attacks. We propose a novel method to protect the property information in speech data by decomposing various properties in the sound and adding perturbations to these properties. Our experiments show that the proposed method offers better privacy-utility trade-offs than existing methods. The trade-offs enable more effective attack prevention while maintaining similar FL utility levels. This work can guide future work on privacy protection methods in speech processing.