Generalized Inverses of Matrix Products: From Fundamental Subspaces to Randomized Decompositions

TL;DR

This paper develops a geometric framework for matrix pseudoinverses, unifies existing formulas, introduces a new randomized inverse formula, and applies these insights to randomized linear algebra algorithms and resistance estimation.

Contribution

It presents a unifying geometric framework for generalized inverses of matrix products, introduces a novel randomized inverse formula, and connects these to practical algorithms and applications.

Findings

Reverse order law holds under independence conditions.

New randomized formula for generalized inverse with rank preservation.

Rigorous bounds for resistance approximation errors.

Abstract

We investigate the Moore-Penrose pseudoinverse and generalized inverse of a matrix product $A=CR$ to establish a unifying framework for generalized and randomized matrix inverses. This analysis is rooted in first principles, focusing on the geometry of the four fundamental subspaces. We examine: (1) the reverse order law, $A^+ = R^+C^+$, which holds when $C$ has independent columns and $R$ has independent rows, (2) the universally correct formula, $A^+ = (C^+CR)^+(CRR^+)^+$, providing a geometric interpretation of the mappings between the involved subspaces, (3) a new generalized randomized formula, $A^+_p = (P^TA)^+P^TAQ(AQ)^+$, which gives $A^+_p = A^+$ if and only if the sketching matrices $P$ and $Q$ preserve the rank of $A$, i.e., $\mathrm{rank}(P^TA) = \mathrm{rank}(AQ) = \mathrm{rank}(A)$. The framework is extended to generalized $\{1,2\}$-inverses and specialized forms, revealing the underlying structure of established randomized linear algebra algorithms, including randomized SVD, the Nystr\"om approximation, and CUR decomposition. We demonstrate applications in sparse sensor placement and effective resistance estimation. For the latter, we provide a rigorous quantitative analysis of an approximation scheme, establishing that it always underestimates the true resistance and deriving a worst-case spectral bound on the error of resistance differences.