A Privacy-Preserving Semantic-Segmentation Method Using Domain-Adaptation Technique

TL;DR

This paper introduces a privacy-preserving semantic segmentation approach that employs perceptual encryption and domain adaptation on Vision Transformer embeddings, maintaining high accuracy comparable to unencrypted models.

Contribution

It presents a novel method combining perceptual encryption with domain adaptation on ViT embeddings for privacy-preserving semantic segmentation.

Findings

Achieves near-original accuracy with encrypted images

Effective domain adaptation on ViT embeddings

Validated on Segmentation Transformer model

Abstract

We propose a privacy-preserving semantic-segmentation method for applying perceptual encryption to images used for model training in addition to test images. This method also provides almost the same accuracy as models without any encryption. The above performance is achieved using a domain-adaptation technique on the embedding structure of the Vision Transformer (ViT). The effectiveness of the proposed method was experimentally confirmed in terms of the accuracy of semantic segmentation when using a powerful semantic-segmentation model with ViT called Segmentation Transformer.