L1 Regression with Lewis Weights Subsampling

TL;DR

This paper introduces a method for approximate $ ext{L}_1$ regression using Lewis weight sampling, achieving high-probability error bounds with fewer labeled samples, improving efficiency over previous methods.

Contribution

It demonstrates that Lewis weight sampling effectively reduces label complexity for $ ext{L}_1$ regression, with better dependence on failure probability $\

Findings

Sampling with Lewis weights achieves near-optimal error with fewer labels.

The method has exponentially better dependence on the failure probability $\

A matching lower bound confirms the near-optimality of the approach.

Abstract

We consider the problem of finding an approximate solution to $\ell_1$ regression while only observing a small number of labels. Given an $n \times d$ unlabeled data matrix $X$, we must choose a small set of $m \ll n$ rows to observe the labels of, then output an estimate $\widehat{\beta}$ whose error on the original problem is within a $1 + \varepsilon$ factor of optimal. We show that sampling from $X$ according to its Lewis weights and outputting the empirical minimizer succeeds with probability $1-\delta$ for $m > O(\frac{1}{\varepsilon^2} d \log \frac{d}{\varepsilon \delta})$. This is analogous to the performance of sampling according to leverage scores for $\ell_2$ regression, but with exponentially better dependence on $\delta$. We also give a corresponding lower bound of $\Omega(\frac{d}{\varepsilon^2} + (d + \frac{1}{\varepsilon^2}) \log\frac{1}{\delta})$.