A New Identity for the Least-square Solution of Overdetermined Set of Linear Equations

TL;DR

This paper introduces a new identity for the least-square solution of overdetermined linear systems, showing that a weighted average of solutions from consistent subsets matches the classical least-square solution.

Contribution

It proposes a novel approach considering all consistent subsets and weighting their solutions, providing a new perspective on least-square solutions.

Findings

Weighted average of subset solutions equals least-square solution

New identity links subset solutions to classical least squares

Method generalizes understanding of overdetermined systems

Abstract

In this paper, we prove a new identity for the least-square solution of an over-determined set of linear equation $Ax=b$, where $A$ is an $m\times n$ full-rank matrix, $b$ is a column-vector of dimension $m$, and $m$ (the number of equations) is larger than or equal to $n$ (the dimension of the unknown vector $x$). Generally, the equations are inconsistent and there is no feasible solution for $x$ unless $b$ belongs to the column-span of $A$. In the least-square approach, a candidate solution is found as the unique $x$ that minimizes the error function $\|Ax-b\|_2$. We propose a more general approach that consist in considering all the consistent subset of the equations, finding their solutions, and taking a weighted average of them to build a candidate solution. In particular, we show that by weighting the solutions with the squared determinant of their coefficient matrix, the resulting candidate solution coincides with the least square solution.