# Improved Convergence for $\ell_\infty$ and $\ell_1$ Regression via Iteratively Reweighted Least Squares

## TL;DR

This paper introduces a simple IRLS algorithm for $\, ext{l}_ ext{infinity}$ and $ ext{l}_1$ regression that converges efficiently with a running time independent of input conditioning, outperforming previous methods.

## Contribution

A new natural IRLS algorithm with provable convergence for $\, ext{l}_ ext{infinity}$ and $ ext{l}_1$ regression, featuring a runtime independent of input conditioning and sublinear in $1/\, ext{ extepsilon}$.

## Key findings

- Converges to a $(1+ ext{ extepsilon})$-approximate solution in specified iterations.
- Runtime is independent of input matrix conditioning.
- Outperforms previous algorithms by at least a factor of $1/ ext{ extepsilon}^2$.

## Abstract

The iteratively reweighted least squares method (IRLS) is a popular technique used in practice for solving regression problems. Various versions of this method have been proposed, but their theoretical analyses failed to capture the good practical performance. In this paper we propose a simple and natural version of IRLS for solving $\ell_\infty$ and $\ell_1$ regression, which provably converges to a $(1+\epsilon)$-approximate solution in $O(m^{1/3}\log(1/\epsilon)/\epsilon^{2/3} + \log m/\epsilon^2)$ iterations, where $m$ is the number of rows of the input matrix. Interestingly, this running time is independent of the conditioning of the input, and the dominant term of the running time depends sublinearly in $\epsilon^{-1}$, which is atypical for the optimization of non-smooth functions. This improves upon the more complex algorithms of Chin et al. (ITCS '12), and Christiano et al. (STOC '11) by a factor of at least $1/\epsilon^2$, and yields a truly efficient natural algorithm for the slime mold dynamics (Straszak-Vishnoi, SODA '16, ITCS '16, ITCS '17).