Explanation of multicollinearity using the decomposition theorem of ordinary linear regression models

TL;DR

This paper explains multicollinearity in multiple linear regression using the decomposition theorem, highlighting geometric interpretations and sample effects, and critiques existing diagnostic methods for their unreliability.

Contribution

It introduces a geometric decomposition approach to analyze multicollinearity and emphasizes the importance of sample structure and size in understanding its causes.

Findings

Univariate coefficients decompose into partial coefficients via the parallelogram rule.

Multicollinearity can be a sample phenomenon influenced by population structure.

Existing diagnostic methods based solely on correlation are unreliable.

Abstract

In a multiple linear regression model, the algebraic formula of the decomposition theorem explains the relationship between the univariate regression coefficient and partial regression coefficient using geometry. It was found that univariate regression coefficients are decomposed into their respective partial regression coefficients according to the parallelogram rule. Multicollinearity is analyzed with the help of the decomposition theorem. It was also shown that it is a sample phenomenon that the partial regression coefficients of important explanatory variables are not significant, but the sign expectation deviation cause may be the population structure between the explained variables and explanatory variables or may be the result of sample selection. At present, some methods of diagnostic multicollinearity only consider the correlation of explanatory variables, so these methods are basically unreliable, and handling multicollinearity is blind before the causes are not distinguished. The increase in the sample size can help identify the causes of multicollinearity, and the difference method can play an auxiliary role.