Oblivious Subspace Injection Is Not Enough for Relative Error

TL;DR

This paper demonstrates that oblivious subspace injection alone cannot guarantee relative error bounds in low-rank approximation and regression, highlighting the need for additional properties for such guarantees.

Contribution

It provides counterexamples showing OSI does not imply relative-error guarantees and identifies the additional property needed for these guarantees.

Findings

OSI alone does not yield relative-error bounds.

Counterexamples for least squares and SVD show limitations of OSI.

Adding control on residuals recovers near-relative-error guarantees.

Abstract

Oblivious subspace injection (OSI) was introduced by Cama\~no, Epperly, Meyer, and Tropp in 2025 as a much weaker sketching property than oblivious subspace embedding (OSE) that still yields constant-factor guarantees for randomized low-rank approximation and sketch-and-solve least-squares regression. At the Simons Institute in Berkeley during a workshop in October 2025, it was asked whether OSIs also imply relative error bounds rather than just constant-factor guarantees. We show that, from a theoretical standpoint, OSI alone does not yield OSE-style relative-error guarantees whose failure probability is controlled solely by the OSI failure parameter, even though OSI sketches often perform extremely well in practice. We provide counterexamples showing this for sketch-and-solve least squares and for randomized SVD in the Frobenius norm. The missing ingredient from a sketch satisfying only OSI is upper control on the optimal residual or tail component, and when one ensures the sketch has this additional property, a near-relative-error bound is recovered. We also show that there is a natural $\ell_p$ analogue of OSI giving constant-factor sketch-and-solve bounds.