Weighted Orthogonal Components Regression Analysis

TL;DR

This paper introduces weighted orthogonal components regression (WOCR), a flexible framework that unifies existing methods like ridge and principal components regression, offering computational efficiency and improved predictive performance.

Contribution

The paper presents WOCR as a general framework that encompasses known methods and introduces a novel weighting scheme based on component-response correlations for better predictions.

Findings

WOCR includes ridge and principal components regression as special cases.

Weighting components by correlation improves predictive accuracy.

WOCR is computationally efficient and adaptable.

Abstract

In the multiple linear regression setting, we propose a general framework, termed weighted orthogonal components regression (WOCR), which encompasses many known methods as special cases, including ridge regression and principal components regression. WOCR makes use of the monotonicity inherent in orthogonal components to parameterize the weight function. The formulation allows for efficient determination of tuning parameters and hence is computationally advantageous. Moreover, WOCR offers insights for deriving new better variants. Specifically, we advocate weighting components based on their correlations with the response, which leads to enhanced predictive performance. Both simulated studies and real data examples are provided to assess and illustrate the advantages of the proposed methods.