Grouped Mixture of Regressions

TL;DR

This paper introduces the Grouped Mixture of Regressions (GMR), a novel extension of finite mixture regression models that incorporates group structure among observations to improve clustering and prediction accuracy.

Contribution

The paper develops a new GMR model that accounts for group structure and provides a fast EM-based algorithm for fitting, enhancing clustering and predictive performance.

Findings

GMR improves clustering accuracy over traditional FMR.

Sharing information within groups enhances prediction quality.

Validated on synthetic and real-world datasets.

Abstract

Finite Mixture of Regressions (FMR) models are among the most widely used approaches in dealing with the heterogeneity among the observations in regression problems. One of the limitations of current approaches is their inability to incorporate group structure in data when available. In some applications, it is desired to cluster groups of observations together rather than the individual ones. In this work, we extend the FMR framework to allow for group structure among observations, and call the resulting model the Grouped Mixture of Regressions (GMR). We derive a fast fitting algorithm based on the Expectation-Maximization (EM) idea. We also show how the group structure can improve prediction by sharing information among members of each group, as reflected in the posterior predictive density under GMR. %that they don't consider clustering the data when there is group structure. In other words, sometimes it is desired to force the algorithm to cluster groups/blocks of observations, instead of individual observations. %In this work, we propose a maximum likelihood approach to cluster groups of observations. We call this algorithm Group Mixture of Regressions (GMR). Expectation Maximization (EM) is employed to maximize the likelihood. Posterior prediction density for predicting new observations is also derived and presented. The performance of the approach is assessed using both synthetic data as well as a real-world example.