$S^{\top\!}S$-SVD via Sketching and the Nearest $S^{\top\!}S$-orthogonal Matrix

TL;DR

This paper introduces the $S^{ op ext{}}S$-SVD, a novel, computationally efficient matrix decomposition based on sketching techniques, which preserves singular values with high probability and aids in analyzing and solving problems related to approximate orthogonality.

Contribution

The paper proposes the $S^{ op ext{}}S$-SVD, a new decomposition that is less costly than the standard SVD and applicable to sketching-based algorithms, with probabilistic guarantees.

Findings

The $S^{ op ext{}}S$-SVD preserves singular values with high probability.

It provides bounds on the orthogonality of sketching-based matrices.

Application to the nearest $S^{ op ext{}}S$-orthogonal matrix problem with probabilistic bounds.

Abstract

Sketching techniques have gained popularity in numerical linear algebra to accelerate the solution of least squares problems. The so-called $\varepsilon$-subspace embedding property of a sketching matrix $S$ has been largely used to characterize the problem residual norm, since the procedure is no longer optimal in terms of the (classical) Frobenius or Euclidean norm. By building on available results on the SVD of the sketched matrix $SA$ derived by Gilbert, Park, and Wakin (Proc. of SPARS-2013), a novel decomposition of $A$, the $S^{\top\!}S$-SVD, is proposed, which \emph{holds} with high probability, and in which the left singular vectors are orthonormal with respect to a (semi-)norm defined by the sketching matrix $S$. The new decomposition is less expensive to compute than the standard SVD, while preserving the singular values with probabilistic confidence.The $S^{\top\!}S$-SVD appears to be the right tool to analyze the quality of several sketching-based techniques in the literature, for which examples are reported. For instance, it is possible to simply bound the distance from (standard) orthogonality of sketching-based orthogonal matrices in state-of-the-art randomized algorithms for QR factorizations. As an application, the classical problem of the nearest orthogonal matrix is generalized to the new $S^{\top\!}S$-orthogonality, and the $S^{\top\!}S$-SVD is used to solve it. Probabilistic bounds on the quality of the solution are also derived.