Generalized Orthogonal Components Regression for High Dimensional Generalized Linear Models

TL;DR

This paper introduces GOCRE, a novel algorithm for high-dimensional generalized linear models that constructs orthogonal components sequentially, overcoming convergence and computational issues of existing methods, and demonstrating improved performance.

Contribution

GOCRE extends PLS for GLMs by sequentially building convergent orthogonal components, addressing issues of divergence and computational inefficiency in existing IRLS-based methods.

Findings

GOCRE outperforms existing methods in simulations.

GOCRE effectively models high-dimensional categorical data.

The method demonstrates strong convergence properties.

Abstract

Here we propose an algorithm, named generalized orthogonal components regression (GOCRE), to explore the relationship between a categorical outcome and a set of massive variables. A set of orthogonal components are sequentially constructed to account for the variation of the categorical outcome, and together build up a generalized linear model (GLM). This algorithm can be considered as an extension of the partial least squares (PLS) for GLMs, but overcomes several issues of existing extensions based on iteratively reweighted least squares (IRLS). First, existing extensions construct a different set of components at each iteration and thus cannot provide a convergent set of components. Second, existing extensions are computationally intensive because of repetitively constructing a full set of components. Third, although they pursue the convergence of regression coefficients, the resultant regression coefficients may still diverge especially when building logistic regression models. GOCRE instead sequentially builds up each orthogonal component upon convergent construction, and simultaneously regresses against these orthogonal components to fit the GLM. The performance of the new method is demonstrated by both simulation studies and a real data example.