A Convex Program for Mixed Linear Regression with a Recovery Guarantee for Well-Separated Data

TL;DR

This paper presents a convex second-order cone program for mixed linear regression that guarantees exact recovery of mixture components under well-separated data conditions, with practical implementation via reweighted least squares.

Contribution

It introduces a novel convex optimization approach for mixed linear regression with theoretical guarantees for exact recovery in noiseless, well-separated data scenarios.

Findings

Exact recovery of mixture components under certain conditions.

The method works with at least d measurements per class.

Iteratively reweighted least squares improves practical performance.

Abstract

We introduce a convex approach for mixed linear regression over $d$ features. This approach is a second-order cone program, based on L1 minimization, which assigns an estimate regression coefficient in $\mathbb{R}^{d}$ for each data point. These estimates can then be clustered using, for example, $k$-means. For problems with two or more mixture classes, we prove that the convex program exactly recovers all of the mixture components in the noiseless setting under technical conditions that include a well-separation assumption on the data. Under these assumptions, recovery is possible if each class has at least $d$ independent measurements. We also explore an iteratively reweighted least squares implementation of this method on real and synthetic data.