PATE-AAE: Incorporating Adversarial Autoencoder into Private Aggregation of Teacher Ensembles for Spoken Command Classification

TL;DR

This paper introduces PATE-AAE, a novel privacy-preserving speech classification framework combining an adversarial autoencoder with the PATE scheme, achieving improved accuracy while maintaining strong differential privacy guarantees.

Contribution

It replaces GAN with an adversarial autoencoder in PATE, enabling better synthetic speech generation for privacy-preserving classification.

Findings

PATE-AAE outperforms PATE-GAN and DP-GAN in accuracy on Google Speech Commands Dataset.

The framework maintains $ ext{ε}$=0.01 differential privacy with improved utility.

Synthetic speech quality benefits from the AAE architecture's discriminative training.

Abstract

We propose using an adversarial autoencoder (AAE) to replace generative adversarial network (GAN) in the private aggregation of teacher ensembles (PATE), a solution for ensuring differential privacy in speech applications. The AAE architecture allows us to obtain good synthetic speech leveraging upon a discriminative training of latent vectors. Such synthetic speech is used to build a privacy-preserving classifier when non-sensitive data is not sufficiently available in the public domain. This classifier follows the PATE scheme that uses an ensemble of noisy outputs to label the synthetic samples and guarantee $\varepsilon$-differential privacy (DP) on its derived classifiers. Our proposed framework thus consists of an AAE-based generator and a PATE-based classifier (PATE-AAE). Evaluated on the Google Speech Commands Dataset Version II, the proposed PATE-AAE improves the average classification accuracy by +$2.11\%$ and +$6.60\%$, respectively, when compared with alternative privacy-preserving solutions, namely PATE-GAN and DP-GAN, while maintaining a strong level of privacy target at $\varepsilon$=0.01 with a fixed $\delta$=10$^{-5}$.