Speech privacy-preserving methods using secret key for convolutional neural network models and their robustness evaluation

TL;DR

This paper introduces secret key-based encryption methods for CNN speech models that preserve privacy without degrading performance when correct keys are used, and significantly reduce accuracy with incorrect keys, enhancing privacy.

Contribution

It proposes novel encryption techniques for speech queries using secret keys that enable privacy-preserving CNN inference without model modification.

Findings

Correct key usage maintains identification accuracy.

Incorrect keys significantly reduce performance.

ROM-based encryption offers high privacy with small key space.

Abstract

In this paper, we propose privacy-preserving methods with a secret key for convolutional neural network (CNN)-based models in speech processing tasks. In environments where untrusted third parties, like cloud servers, provide CNN-based systems, ensuring the privacy of speech queries becomes essential. This paper proposes encryption methods for speech queries using secret keys and a model structure that allows for encrypted queries to be accepted without decryption. Our approach introduces three types of secret keys: Shuffling, Flipping, and random orthogonal matrix (ROM). In experiments, we demonstrate that when the proposed methods are used with the correct key, identification performance did not degrade. Conversely, when an incorrect key is used, the performance significantly decreased. Particularly, with the use of ROM, we show that even with a relatively small key space, high privacy-preserving performance can be maintained many speech processing tasks. Furthermore, we also demonstrate the difficulty of recovering original speech from encrypted queries in various robustness evaluations.