Linear Regression from 1-bit Quantized Data

TL;DR

This paper develops a new estimator for linear regression using 1-bit quantized data, providing theoretical error bounds and asymptotic distribution results, with extensions to high-dimensional settings.

Contribution

It introduces a plug-in based estimator for 1-bit quantized linear regression and analyzes its error bounds, efficiency, and asymptotic properties, including high-dimensional extensions.

Findings

Non-asymptotic $\, ext{l}_2$-error bounds established.

Asymptotic distribution derived for fixed number of predictors.

Numerical experiments validate theoretical results.

Abstract

Motivated by the prevalence of environments in which data is abundant while resources for storage and/or transmission might be scarce, we study linear regression when predictors, their squares, and responses are subject to single-bit dithered quantization. An estimator relying on plug-in estimation of the quadratic and linear terms in the quadratic program formulation of the least squares problem is proposed. We provide a non-asymptotic bound on the $\ell_2$-estimation error of this estimator and obtain its asymptotic distribution when the number of predictors is fixed, which can be used for inference and an investigation of the mean-square error efficiency relative to the ordinary least squares estimator. It is shown that for the quantization protocol under consideration, substantial improvements over the proposed estimator cannot be expected. A compression pipeline in which the underlying data is first subject to sketching and subsequently quantization can be studied within our framework as well. We also present an extension to address high-dimensional predictors. Numerical experiments with synthetic data complement our theoretical findings.