Human-imperceptible, Machine-recognizable Images

TL;DR

This paper introduces a privacy-preserving learning method where images are encrypted to be human-imperceptible yet still recognizable by machines, enabling effective vision tasks without exposing sensitive data.

Contribution

It proposes a novel encryption strategy and minimal model adaptations allowing vision transformers to learn from encrypted images, ensuring privacy and maintaining accuracy.

Findings

Achieves comparable accuracy to traditional methods on ImageNet and COCO.

Encrypted images are intractable to decrypt by attackers, including vision transformer-based ones.

Ensures privacy by making images human-imperceptible while machine-recognizable.

Abstract

Massive human-related data is collected to train neural networks for computer vision tasks. A major conflict is exposed relating to software engineers between better developing AI systems and distancing from the sensitive training data. To reconcile this conflict, this paper proposes an efficient privacy-preserving learning paradigm, where images are first encrypted to become ``human-imperceptible, machine-recognizable'' via one of the two encryption strategies: (1) random shuffling to a set of equally-sized patches and (2) mixing-up sub-patches of the images. Then, minimal adaptations are made to vision transformer to enable it to learn on the encrypted images for vision tasks, including image classification and object detection. Extensive experiments on ImageNet and COCO show that the proposed paradigm achieves comparable accuracy with the competitive methods. Decrypting the encrypted images requires solving an NP-hard jigsaw puzzle or an ill-posed inverse problem, which is empirically shown intractable to be recovered by various attackers, including the powerful vision transformer-based attacker. We thus show that the proposed paradigm can ensure the encrypted images have become human-imperceptible while preserving machine-recognizable information. The code is available at \url{https://github.com/FushengHao/PrivacyPreservingML.}