HawkEye: Statically and Accurately Profiling the Communication Cost of Models in Multi-party Learning

TL;DR

HawkEye is a static profiling framework that accurately estimates the communication costs of models in multi-party learning, eliminating the need for dynamic profiling during secure training or inference.

Contribution

HawkEye introduces a static analysis method using prefix structures and automatic differentiation to efficiently and accurately profile communication costs in MPL models.

Findings

HawkEye's static profiling closely matches dynamic profiling results.

The framework reduces human effort in identifying communication bottlenecks.

HawkEye works effectively across multiple MPL frameworks.

Abstract

Multi-party computation (MPC) based machine learning, referred to as multi-party learning (MPL), has become an important technology for utilizing data from multiple parties with privacy preservation. In recent years, in order to apply MPL in more practical scenarios, various MPC-friendly models have been proposedto reduce the extraordinary communication overhead of MPL. Within the optimization of MPC-friendly models, a critical element to tackle the challenge is profiling the communication cost of models. However, the current solutions mainly depend on manually establishing the profiles to identify communication bottlenecks of models, often involving burdensome human efforts in a monotonous procedure. In this paper, we propose HawkEye, a static model communication cost profiling framework, which enables model designers to get the accurate communication cost of models in MPL frameworks without dynamically running the secure model training or inference processes on a specific MPL framework. Firstly, to profile the communication cost of models with complex structures, we propose a static communication cost profiling method based on a prefix structure that records the function calling chain during the static analysis. Secondly, HawkEye employs an automatic differentiation library to assist model designers in profiling the communication cost of models in PyTorch. Finally, we compare the static profiling results of HawkEye against the profiling results obtained through dynamically running secure model training and inference processes on five popular MPL frameworks, CryptFlow2, CrypTen, Delphi, Cheetah, and SecretFlow-SEMI2K. The experimental results show that HawkEye can accurately profile the model communication cost without dynamic profiling.