Block-Wise Encryption for Reliable Vision Transformer models

TL;DR

This paper introduces a block-wise image encryption method for vision transformers that preserves model performance while enhancing privacy and access control, applicable to federated learning without network modifications.

Contribution

The authors propose a novel block-wise encryption scheme for vision transformers that maintains accuracy and enables easy key updates, addressing performance issues of conventional perceptual encryption.

Findings

Achieves comparable performance to unencrypted models on CIFAR datasets

Provides effective access control and privacy preservation

No need for network modifications or retraining

Abstract

This article presents block-wise image encryption for the vision transformer and its applications. Perceptual image encryption for deep learning enables us not only to protect the visual information of plain images but to also embed unique features controlled with a key into images and models. However, when using conventional perceptual encryption methods, the performance of models is degraded due to the influence of encryption. In this paper, we focus on block-wise encryption for the vision transformer, and we introduce three applications: privacy-preserving image classification, access control, and the combined use of federated learning and encrypted images. Our scheme can have the same performance as models without any encryption, and it does not require any network modification. It also allows us to easily update the secret key. In experiments, the effectiveness of the scheme is demonstrated in terms of performance degradation and access control on the CIFAR10 and CIFAR-100 datasets.