Multi-Randomized Kaczmarz for Latent Class Regression

TL;DR

This paper introduces a novel iterative algorithm based on the randomized Kaczmarz method that automatically identifies subgroups within data and performs linear regression on these groups, capturing diverse effects.

Contribution

It presents a new subgroup-aware linear regression algorithm with proven convergence properties, enhancing interpretability and subgroup detection in data analysis.

Findings

Proves almost sure convergence of the method

Demonstrates linear convergence in expectation under certain conditions

Successfully identifies multiple data trends in simulated experiments

Abstract

Linear regression is effective at identifying interpretable trends in a data set, but averages out potentially different effects on subgroups within data. We propose an iterative algorithm based on the randomized Kaczmarz (RK) method to automatically identify subgroups in data and perform linear regression on these groups simultaneously. We prove almost sure convergence for this method, as well as linear convergence in expectation under certain conditions. The result is an interpretable collection of different weight vectors for the regressor variables that capture the different trends within data. Furthermore, we experimentally validate our convergence results by demonstrating the method can successfully identify two trends within simulated data.