Householder Meets Student
John H. Elton, Andrew B. Gardner

TL;DR
This paper introduces a simple formula derived from the Householder QR algorithm that provides a concrete, computationally efficient way to generate independent residuals in linear regression, offering new proofs and insights.
Contribution
It presents a novel, simple formula for residuals in linear regression using QR factorization, enabling the generation of independent residuals and providing a new proof of Student's theorem.
Findings
New simple formula for residuals in regression
Generation of independent residuals with same sum of squares
Connection to Cochran's theorem
Abstract
The Householder algorithm for the QR factorization of a tall thin n x p full-rank matrix X has the added bonus of producing a matrix M with orthonormal columns that are a basis for the orthocomplement of the column space of X. We give a simple formula for M-Transpose x when x is in that orthocomplement. The formula does not require computing M, it only requires the R factor of a QR factorization. This is used to get a remarkably simple computable concrete representation of independent "residuals" in classical linear regression. For Students problem, when p=1, if R(j)=Y(j)-Ybar are the usual (non-independent) residuals, W(j)=R(j+1) - R(1)/(sqrt(n)+1) gives n-1 i.i.d. mean-zero normal variables whose sum of squares is the same as that of the n residuals. Those properties of this formula can (in hindsight) easily be verified directly, yielding a new simple and concrete proof of Student's…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMatrix Theory and Algorithms · Advanced Optimization Algorithms Research · Sparse and Compressive Sensing Techniques
