Harmonic Coding: An Optimal Linear Code for Privacy-Preserving Gradient-Type Computation

TL;DR

Harmonic Coding introduces an optimal linear coding scheme for privacy-preserving distributed gradient computations, improving efficiency and universality over prior methods by leveraging harmonic progression for redundancy and decoding.

Contribution

The paper presents Harmonic Coding, a universal and optimal coding scheme for gradient-type functions that outperforms existing methods like Shamir's secret sharing and Lagrange Coded Computing.

Findings

Harmonic Coding achieves minimal worker requirements for universal gradient computation.

It provides a strict improvement over prior schemes by injecting harmonic redundancy.

The scheme's optimality is proven through a matching converse bound.

Abstract

We consider the problem of distributedly computing a general class of functions, referred to as gradient-type computation, while maintaining the privacy of the input dataset. Gradient-type computation evaluates the sum of some `partial gradients', defined as polynomials of subsets of the input. It underlies many algorithms in machine learning and data analytics. We propose Harmonic Coding, which universally computes any gradient-type function, while requiring the minimum possible number of workers. Harmonic Coding strictly improves computing schemes developed based on prior works, such as Shamir's secret sharing and Lagrange Coded Computing, by injecting coded redundancy using harmonic progression. It enables the computing results of the workers to be interpreted as the sum of partial gradients and some redundant results, which then allows the cancellation of non-gradient terms in the decoding process. By proving a matching converse, we demonstrate the optimality of Harmonic Coding, even compared to the schemes that are non-universal (i.e., can be designed based on a specific gradient-type function).