Training Machine Learning Models on Encrypted Data: A Privacy-Preserving Framework using Homomorphic Encryption

TL;DR

This paper proposes a framework for training machine learning models on encrypted data using homomorphic encryption, demonstrating feasibility with models like KNN and linear regression while addressing privacy and computational challenges.

Contribution

It introduces a proof-of-concept framework leveraging CKKS homomorphic encryption for privacy-preserving ML training and inference, showing comparable performance to plaintext models.

Findings

Models trained on encrypted data achieve similar accuracy to plaintext models.

Homomorphic encryption enables privacy-preserving inference for neural networks.

Computational overhead and noise management remain significant challenges.

Abstract

The use of Machine Learning (ML) for data-driven decision-making often relies on access to sensitive datasets, which introduces privacy challenges. Traditional encryption methods protect data at rest or in transit but fail to secure it during processing, exposing it to unauthorized access. Homomorphic encryption emerges as a transformative solution, enabling computations on encrypted data without decryption, thus preserving confidentiality throughout the ML pipeline. This paper addresses the challenge of training ML models on encrypted data while maintaining accuracy and efficiency by proposing a proof-of-concept for a privacy-preserving framework that leverages Cheon-Kim-Kim-Song (CKKS) for approximate real-number arithmetic. Also, it demonstrates the feasibility of training K-Nearest Neighbors (KNN) and linear regression models on encrypted data, and evaluates encrypted inference for a basic Multilayer Perceptron (MLP) architecture. Experimental results show that models trained under Homomorphic encryption achieve performance metrics comparable to plaintext-trained models, validating the approach. However, challenges such as computational overhead, noise management, and limited support for non-polynomial operations persist. This work lays the groundwork for broader adoption of privacy-preserving ML in real-world applications, balancing security with computational feasibility.