Incremental Seeded EM Algorithm for Clusterwise Linear Regression

TL;DR

This paper introduces an Incremental Seeded EM algorithm that enhances clusterwise linear regression, especially in complex scenarios, and presents new concepts for evaluating model quality without ground truth.

Contribution

The paper proposes a novel Incremental Seeded EM algorithm and introduces the concepts of Resolvability and X-predictability for better model assessment.

Findings

Improved performance in high-dimensional, noisy, and large-cluster scenarios.

Resolvability index correlates strongly with model quality.

New metrics enable model evaluation without ground truth.

Abstract

This paper proposes Incremental Seeded Expectation Maximization, an algorithm that improves upon the traditional Expectation Maximization computational flow for clusterwise or finite mixture linear regression tasks. The proposed method shows significantly better performance, particularly in scenarios involving high-dimensional input, noisy data, or a large number of clusters. Alongside the new algorithm, this paper introduces the concepts of $\textit{Resolvability}$ and $\textit{X-predictability}$, which enable more rigorous discussions of clusterwise regression problems. The resolvability index is quantified using parameters derived from the model, and results demonstrate its strong connection to model quality without requiring knowledge of the ground truth. This makes the $\textit{Resolvability}$ especially useful for assessing the quality of clusterwise regression models, and by extension, the conclusions drawn from them.