Solving The Ordinary Least Squares in Closed Form, Without Inversion or Normalization

TL;DR

This paper introduces a novel method to compute ordinary least squares estimates directly in closed form by linking LU factorization and Gram-Schmidt orthogonalization without normalization, avoiding explicit matrix inversion.

Contribution

It presents a new closed-form solution for OLS coefficients that leverages LU and Gram-Schmidt factorizations without normalization, enabling direct and iterative computation.

Findings

Coefficients expressed as linear combinations of Gram-Schmidt vectors and data matrix.

Avoids explicit matrix inversion in OLS computation.

Allows iterative calculation of coefficients using backward or forward algorithms.

Abstract

By connecting the LU factorization and the Gram-Schmidt orthogonalization without any normalization, closed-forms for the coefficients of the ordinary least squares estimates are presented. Instead of using matrix inversion explicitly, each of the coefficients is expressed and computed directly as a linear combination of non-normalized Gram-Schmidt vectors and the original data matrix and also in terms of the upper triangular factor from LU factorization. The coefficients may computed iteratively using backward or forward algorithms given.