Orthonormal Sketches for Secure Coded Regression

TL;DR

This paper introduces a secure, distributed linear regression method using randomized orthonormal sketches, enhancing speed, security, and straggler resilience in asynchronous systems with practical numerical validation.

Contribution

It presents a novel approach combining randomized orthonormal matrices and block subsampling for secure, efficient distributed regression, extending the Subsampled Randomized Hadamard Transform.

Findings

Improved regression speed in distributed settings.

Enhanced security through orthonormal transformations.

Demonstrated effectiveness via numerical experiments.

Abstract

In this work, we propose a method for speeding up linear regression distributively, while ensuring security. We leverage randomized sketching techniques, and improve straggler resilience in asynchronous systems. Specifically, we apply a random orthonormal matrix and then subsample in \textit{blocks}, to simultaneously secure the information and reduce the dimension of the regression problem. In our setup, the transformation corresponds to an encoded encryption in an \textit{approximate} gradient coding scheme, and the subsampling corresponds to the responses of the non-straggling workers; in a centralized coded computing network. We focus on the special case of the \textit{Subsampled Randomized Hadamard Transform}, which we generalize to block sampling; and discuss how it can be used to secure the data. We illustrate the performance through numerical experiments.