Privacy-Preserving Chest X-ray Classification in Latent Space with Homomorphically Encrypted Neural Inference

TL;DR

This paper introduces a privacy-preserving framework for chest X-ray classification using homomorphic encryption and image compression, enabling secure inference with reduced computational load.

Contribution

It proposes a novel HE inference method employing VQGAN compression and polynomial activation approximation to make encrypted medical image analysis more practical.

Findings

Achieved significant reduction in computational cost with image compression

Maintained high classification accuracy despite encryption constraints

Demonstrated feasibility on real chest X-ray datasets

Abstract

Medical imaging data contain sensitive patient information requiring strong privacy protection. Many analytical setups require data to be sent to a server for inference purposes. Homomorphic encryption (HE) provides a solution by allowing computations to be performed on encrypted data without revealing the original information. However, HE inference is computationally expensive, particularly for large images (e.g., chest X-rays). In this study, we propose an HE inference framework for medical images that uses VQGAN to compress images into latent representations, thereby significantly reducing the computational burden while preserving image quality. We approximate the activation functions with lower-degree polynomials to balance the accuracy and efficiency in compliance with HE requirements. We observed that a downsampling factor of eight for compression achieved an optimal balance between performance and computational cost. We further adapted the squeeze and excitation module, which is known to improve traditional CNNs, to enhance the HE framework. Our method was tested on two chest X-ray datasets for multi-label classification tasks using vanilla CNN backbones. Although HE inference remains relatively slow and introduces minor performance differences compared with unencrypted inference, our approach shows strong potential for practical use in medical images