A globally convergent algorithm for lasso-penalized mixture of linear regression models

TL;DR

This paper introduces a new globally convergent algorithm for the lasso-penalized mixture of linear regression models, improving upon previous methods by ensuring monotonic convergence and addressing mixture proportion updates via polynomial root finding.

Contribution

A novel minorization-maximization algorithm for L-MLR that guarantees global convergence and simplifies mixture proportion updates through polynomial root solving.

Findings

Algorithm demonstrates convergence in simulations

Effective in real data application to baseball salaries

Outperforms previous approximate EM algorithms

Abstract

Variable selection is an old and pervasive problem in regression analysis. One solution is to impose a lasso penalty to shrink parameter estimates toward zero and perform continuous model selection. The lasso-penalized mixture of linear regressions model (L-MLR) is a class of regularization methods for the model selection problem in the fixed number of variables setting. In this article, we propose a new algorithm for the maximum penalized-likelihood estimation of the L-MLR model. This algorithm is constructed via the minorization--maximization algorithm paradigm. Such a construction allows for coordinate-wise updates of the parameter components, and produces globally convergent sequences of estimates that generate monotonic sequences of penalized log-likelihood values. These three features are missing in the previously presented approximate expectation-maximization algorithms. The previous difficulty in producing a globally convergent algorithm for the maximum penalized-likelihood estimation of the L-MLR model is due to the intractability of finding exact updates for the mixture model mixing proportions in the maximization-step. In our algorithm, we solve this issue by showing that it can be converted into a polynomial root finding problem. Our solution to this problem involves a polynomial basis conversion that is interesting in its own right. The method is tested in simulation and with an application to Major League Baseball salary data from the 1990s and the present day. We explore the concept of whether player salaries are associated with batting performance.