Design and Optimization of Cloud Native Homomorphic Encryption Workflows for Privacy-Preserving ML Inference

TL;DR

This paper introduces a cloud native framework for homomorphic encryption workflows that significantly accelerates privacy-preserving machine learning inference while reducing resource usage, enabling practical secure ML deployment in cloud environments.

Contribution

It presents a novel integrated architecture combining containerized HE modules with Kubernetes orchestration and optimization strategies to enhance efficiency and scalability of privacy-preserving ML inference.

Findings

Achieves up to 3.2x inference acceleration

Reduces memory utilization by 40%

Demonstrates practical deployment of secure ML-as-a-Service

Abstract

As machine learning (ML) models become increasingly deployed through cloud infrastructures, the confidentiality of user data during inference poses a significant security challenge. Homomorphic Encryption (HE) has emerged as a compelling cryptographic technique that enables computation on encrypted data, allowing predictions to be generated without decrypting sensitive inputs. However, the integration of HE within large scale cloud native pipelines remains constrained by high computational overhead, orchestration complexity, and model compatibility issues. This paper presents a systematic framework for the design and optimization of cloud native homomorphic encryption workflows that support privacy-preserving ML inference. The proposed architecture integrates containerized HE modules with Kubernetes-based orchestration, enabling elastic scaling and parallel encrypted computation across distributed environments. Furthermore, optimization strategies including ciphertext packing, polynomial modulus adjustment, and operator fusion are employed to minimize latency and resource consumption while preserving cryptographic integrity. Experimental results demonstrate that the proposed system achieves up to 3.2times inference acceleration and 40% reduction in memory utilization compared to conventional HE pipelines. These findings illustrate a practical pathway for deploying secure ML-as-a-Service (MLaaS) systems that guarantee data confidentiality under zero-trust cloud conditions.