Study of Compressed Randomized UTV Decompositions for Low-Rank Matrix Approximations in Data Science

TL;DR

This paper introduces CoR-UTV, a novel randomized matrix decomposition method that efficiently computes low-rank approximations of large matrices with fewer data passes, suitable for modern computational platforms.

Contribution

The paper proposes the CoR-UTV algorithm, a new randomized low-rank matrix decomposition technique with variants, improving efficiency and robustness in data science applications.

Findings

CoR-UTV requires only a few passes over the data.

It runs in $O(mnk)$ operations, suitable for large matrices.

Simulations show CoR-UTV outperforms existing methods.

Abstract

In this work, a novel rank-revealing matrix decomposition algorithm termed Compressed Randomized UTV (CoR-UTV) decomposition along with a CoR-UTV variant aided by the power method technique is proposed. CoR-UTV computes an approximation to a low-rank input matrix by making use of random sampling schemes. Given a large and dense matrix of size $m\times n$ with numerical rank $k$, where $k \ll \text{min} \{m,n\}$, CoR-UTV requires a few passes over the data, and runs in $O(mnk)$ floating-point operations. Furthermore, CoR-UTV can exploit modern computational platforms and can be optimized for maximum efficiency. CoR-UTV is also applied for solving robust principal component analysis problems. Simulations show that CoR-UTV outperform existing approaches.

Tables

Table 1. Table 1 : Numerical results for synthetic matrix recovery.

$n$	$r\left(𝐋\right)$	${\Vert 𝐒\Vert }_0$	Methods	$r\left({𝐋}^{\ast }\right)$	${\Vert {𝐒}^{\ast }\Vert }_0$	Time(s)	Iter.	$\zeta$
1000	50	5e4	InexactALM 50 5e4 4.1 12 2.1e-6 ALM-CoRUTV 50 5e4 0.6 12 9.6e-6
2000	100	2e5	InexactALM 100 2e5 27.4 12 2.7e-6 ALM-CoRUTV 100 2e5 3.7 12 8.3e-6
3000	150	45e4	InexactALM 150 45e4 75.6 12 3.1e-6 ALM-CoRUTV 150 45e4 9.4 12 8.7e-6

Equations

$${\hat{\mathbf{A}}}_{\text{CoR}}={\mathbf{UTV}}^T,$$

$$\mathbf{A}=$$

$$\Vert \mathbf{A}-{\mathbf{A}}_k{\Vert }_2={\sigma }_{k+1},$$

$$\Vert \mathbf{A}-{\mathbf{A}}_k{\Vert }_F=\sqrt{{\sigma }_{k+1}^2+...+{\sigma }_n^2},$$

$${\mathbf{C}}_1=\mathbf{A}\mathbf{\Psi },$$

$${\mathbf{C}}_2={\mathbf{A}}^T{\mathbf{C}}_1,$$

$${\mathbf{C}}_1={\mathbf{Q}}_1{\mathbf{R}}_1,\text{and}{\mathbf{C}}_2={\mathbf{Q}}_2{\mathbf{R}}_2,$$

$$\mathbf{D}={\mathbf{Q}}_1^T\mathbf{A}{\mathbf{Q}}_2,$$

$$\mathbf{D}=\tilde{\mathbf{Q}}\tilde{\mathbf{R}}{\tilde{\mathbf{P}}}^T.$$

$${\hat{\mathbf{A}}}_{\text{CoR}}={\mathbf{UTV}}^T,$$

$$C_{\text{CoR-UTV}}=O(mn\ell ).$$

$${\text{minimize}}_{(\mathbf{L},\mathbf{S})}\Vert \mathbf{L}{\Vert }_{\ast }+\lambda \Vert \mathbf{S}{\Vert }_1$$

$$\text{subject to}\mathbf{M}=\mathbf{L}+\mathbf{S},$$

$${\mathcal{C}}_{\delta }(\mathbf{B})=\mathbf{U}(:,1:r)\mathbf{T}(1:r,:){\mathbf{V}}^T,$$

$$e_k=\Vert \mathbf{A}-{\hat{\mathbf{A}}}_{\text{out}}{\Vert }_F.$$

$${\text{minimize}}_{(\mathbf{L},\mathbf{S})}\Vert \mathbf{L}{\Vert }_{\ast }+\lambda \Vert \mathbf{S}{\Vert }_1$$

$$\text{subject to}\mathbf{M}=\mathbf{L}+\mathbf{S},$$

Full text

**Index Terms— ** Rank-revealing decompositions, low-rank approximations, randomized algorithms, robust PCA.

1 Introduction

Low-rank matrix approximations play an increasingly important role in signal processing and its applications. Such compact representations which retain the key features of a high-dimensional matrix provide a significant reduction in memory requirements, and more importantly, computational costs when the latter scales, e.g., according to a high-degree polynomial, with the dimensionality. Matrices with low-rank structures have found many applications in background subtraction [1, 2], system identification [3], IP network anomaly detection [4, 5], latent variable graphical modeling [6], subspace clustering [7, 8] and sensor and multichannel signal processing [9, 10, 11, 12, 13, 14, 15], [16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27]. .

Singular value decomposition (SVD) [28] and the rank-revealing QR (RRQR) decomposition [29, 30] are among the most commonly used algorithms for computing a low-rank approximation of a matrix. On the other hand, a UTV decomposition [31] is a compromise between the SVD and the RRQR decomposition with the virtues of both: UTV (i) is more efficient than the SVD, and (ii) provides information on the numerical null space of the matrix [31]. Given a matrix $\bf A$, the UTV algorithm computes a decomposition ${\bf A=UTV}^{T}$, where ${\bf U}$ and ${\bf V}$ have orthonormal columns, and ${\bf T}$ is triangular (either lower or upper triangular). These deterministic algorithms, however, are computationally expensive for large data sets. Furthermore, standard techniques for their computation are challenging to parallelize in order to utilize advanced computer architectures [32, 33]. Recently developed algorithms for low-rank approximations based on random sampling schemes, however, have been shown to be remarkably efficient, highly accurate and robust, and are known to outperform existing algorithms in many practical situations [34, 35, 32, 33]. The power of randomized algorithms lies in that (i) they are computationally efficient, and (ii) their main operations can be optimized for maximum efficiency on modern architectures.

This work presents a novel randomized rank-revealing method termed compressed randomized UTV (CoR-UTV) decomposition. Given a large and dense rank-$k$ matrix ${\bf A}\in\mathbb{R}^{m\times n}$, CoR-UTV computes a low-rank approximation $\hat{\bf A}_{\text{CoR}}$ of $\bf A$ such that

[TABLE]

where ${\bf U}$ and ${\bf V}$ have orthonormal columns, and ${\bf T}$ is triangular (either lower or upper, whichever is preferred). CoR-UTV only requires a few passes through data, for a matrix stored externally, and runs in $O(mnk)$ floating-point operations (flops). The operations of CoR-UTV involve matrix-matrix multiplication, the QR and RRQR decompositions. Due to recently developed Communication-Avoiding QR algorithms [36, 37, 38], which can perform the computations with optimal/minimum communication costs, CoR-UTV can be optimized for peak machine performance on modern architectures. We illustrate, through numerical examples, that CoR-UTV is rank-revealer and provides a highly accurate low-rank approximation to a given matrix. Furthermore, we apply CoR-UTV to solve the robust principal component analysis (robust PCA) problem [39, 40], i.e., to decompose a given matrix with grossly corrupted entries into a low-rank matrix plus a sparse matrix of outliers.

The rest of this paper is structured as follows. In Section 2, we introduce the mathematical model of the data and discuss related works. In Section 3, we describe the proposed CoR-UTV method in detail. In Section 4, we develop an algorithm for robust PCA using CoR-UTV. In Section 5, we present and discuss simulation results and conclusions are given in Section 6.

2 Mathematical Model and Related Works

Given a matrix ${\bf A}\in\mathbb{R}^{m\times n}$, where $m\geq n$, with numerical rank $k$, its SVD [28] is defined as:

[TABLE]

where ${\bf U}_{k}\in\mathbb{R}^{m\times k}$, ${\bf U}_{0}\in\mathbb{R}^{m\times n-k}$, ${\bf V}_{k}\in\mathbb{R}^{n\times k}$ and ${\bf V}_{0}\in\mathbb{R}^{n\times n-k}$ have orthonormal columns, ${\bf\Sigma}_{k}\in\mathbb{R}^{k\times k}$ and ${\bf\Sigma}_{0}\in\mathbb{R}^{n-k\times n-k}$ are diagonal matrices containing the singular values, i.e., ${\bf\Sigma}_{k}=\text{diag}(\sigma_{1},...,\sigma_{k})$ and ${\bf\Sigma}_{0}=\text{diag}(\sigma_{k+1},...,\sigma_{n})$. $\bf A$ can be written as ${\bf A}={\bf A}_{k}+{\bf A}_{0}$, where ${\bf A}_{k}={\bf U}_{k}{\bf\Sigma}_{k}{\bf V}_{k}^{T}$, and ${\bf A}_{0}={\bf U}_{0}{\bf\Sigma}_{0}{\bf V}_{0}^{T}$. The SVD constructs the optimal rank-$k$ approximation ${\bf A}_{k}$ to ${\bf A}$, [28] i.e.,

[TABLE]

where ${\|{\cdot}\|_{2}}$ and ${\|{\cdot}\|_{F}}$ denote the spectral norm and the Frobenius norm, respectively. In this paper we focus on the matrix $\bf A$ defined.

The SVD is highly accurate for computing singular subspaces and singular values. However, its computation is costly for large data sets. Moreover, standard techniques for its computation are challenging to parallelize in order to take advantage of modern processors [32, 33]. An economical version of the SVD is the partial SVD based on Krylov subspace methods, such as the Lanczos and Arnoldi algorithms, which constructs an approximate SVD of an input matrix, for instance $\bf A$, at a cost $O(mnk)$. However, the partial SVD suffers from two drawbacks: (i) it is numerically unstable [41, 28], and (ii) it does not lend itself to parallel implementations [32, 33], which makes it unsuitable for modern architectures. Other approaches for low-rank matrix approximations include the RRQR [29] and the UTV decompositions [31]. Even though the QR with column pivoting (QRCP) and UTV decompositions provide highly accurate approximations to $\bf A$, they suffer from two drawbacks: (i) they are costly, i.e., $O(mn^{2})$, and (ii) methods for their computation are challenging to parallelize and hence they cannot exploit modern computational platforms [32, 33].

Recently developed algorithms for low-rank approximations based on randomization [34, 35, 32, 33, 42] have attracted significant attention. The randomized algorithms project a large input matrix onto a lower dimensional space using a random matrix, and apply deterministic methods on the smaller matrix to give an approximation of the matrix. Hence (i) they are computationally efficient, and (ii) lend themselves to parallel implementation. Halko et al. [32] proposed randomized SVD (R-SVD) in which a smaller matrix is formed by linear combinations of columns of the given matrix. The low-rank approximation is then given through the SVD of a reduced-size matrix. Gu [33] applied a slightly modified version of the R-SVD algorithm to improve subspace iteration methods, and presents a new error analysis. Another algorithm proposed in [32, Section 5.5], which we call two-sided randomized SVD (TSR-SVD), is a single-pass method, i.e., it required only one pass through data. It captures most attributes of the data by means of forming the smaller matrix through linear combinations of both rows and columns of the given matrix, and then applies the SVD for further computations. The work in [43] proposed a randomized algorithm termed subspace-orbit randomized SVD (SOR-SVD). SOR-SVD alternately projects the matrix onto its column and row space. The matrix is then transformed into a lower dimensional space, and a truncated SVD follows in order to construct an approximation.

TSR-SVD gives poor approximation compared to the optimal SVD due to the single-pass strategy. SOR-SVD has shown better performance than TSR-SVD, however both methods apply the SVD on the reduced-size matrix. This computation may be burdensome in terms of communication cost [36] for large matrices. In this work, we develop a randomized algorithm for low-rank approximation that with comparable flops (i) outperforms the TSR-SVD in terms of accuracy, and (ii) can utilize advanced computer architectures better than TSR-SVD as well as SOR-SVD.

3 Compressed Randomized UTV Decompositions

In this section, we present a randomized rank-revealing decomposition algorithm termed compressed randomized UTV (CoR-UTV) decomposition [44], which computes a low-rank approximation of a given matrix. We focus on the matrix $\bf A$ with $m\geq n$, where CoR-UTV, in the form of (1), produces an upper triangular middle matrix $\bf T$. The modifications required for a CoR-UTV for the case $m<n$ that produces a lower triangular middle matrix $\bf T$ is straightforward.

3.1 Proposed CoR-UTV Decompositions

Given the matrix ${\bf A}$ and an integer $k\leq\ell<\text{min}\{m,n\}$, the basic version of CoR-UTV is computed as follows: using a random number generator, we form a matrix ${\bf\Psi}\in\mathbb{R}^{n\times\ell}$ with entries drawn independent, identically distributed (i.i.d.) from the standard Gaussian distribution. We then compute the matrix product:

[TABLE]

where ${\bf C}_{1}\in\mathbb{R}^{m\times\ell}$ is, in fact, a projection onto the subspace spanned by columns of ${\bf A}$. Having ${\bf C}_{1}$, we form ${\bf C}_{2}\in\mathbb{R}^{n\times\ell}$:

[TABLE]

where ${\bf C}_{2}$ is, in fact, a projection onto the subspace spanned by rows of ${\bf A}$. Using a QR decomposition, we factor ${\bf C}_{1}$ and ${\bf C}_{2}$ such that:

[TABLE]

where ${\bf Q}_{1}$ and ${\bf Q}_{2}$ are approximate bases for $\mathcal{R}({\bf A})$ and $\mathcal{R}({\bf A}^{T})$, respectively. We now compress $\bf A$ by left and right multiplications by the orthonormal bases obtained, forming the matrix ${\bf D}\in\mathbb{R}^{\ell\times\ell}$:

[TABLE]

We then compute a QRCP of ${\bf D}$:

[TABLE]

The CoR-UTV-based low-rank approximation of $\bf A$ is given by

[TABLE]

where ${\bf U}={\bf Q}_{1}\widetilde{\bf Q}\in\mathbb{R}^{m\times\ell}$ and ${\bf V}={\bf Q}_{2}\widetilde{\bf P}\in\mathbb{R}^{n\times\ell}$ construct approximations to the $\ell$ leading left and right singular vectors of $\bf A$, respectively, and ${\bf T}=\widetilde{\bf R}\in\mathbb{R}^{\ell\times\ell}$ is upper triangular with diagonals approximating the first $\ell$ singular values of $\bf A$.

CoR-UTV requires three passes through data, for a matrix stored externally, but it can be altered to revisit the data only once. To this end, the compressed matrix $\bf D$ (7) can be approximated as follows: both sides of the currently unknown ${\bf D}={\bf Q}_{1}^{T}{\bf A}{\bf Q}_{2}$ are postmultiplied by ${\bf Q}_{2}^{T}{\bf\Psi}$. Having defined ${\bf A}\approx{\bf A}{\bf Q}_{2}{\bf Q}_{2}^{T}$ and ${\bf C}_{1}={\bf A}{\bf\Psi}$, then ${\bf D}_{\text{approx}}={\bf Q}_{1}^{T}{\bf C}_{1}({\bf Q}_{2}^{T}{\bf\Psi})^{\dagger}$.

CoR-UTV is accurate for matrices whose singular values display some decay, however in applications where the data matrix has a slowly decaying singular spectrum, it may produce a poor approximation compared to the SVD. Thus, we incorporate $q$ steps of a power iteration [35, 32] to improve the accuracy of the algorithm in these circumstances. Given the matrix ${\bf A}$, and integers $k\leq\ell<n$ and $q$, the resulting algorithm is described in Alg. 1.

3.2 Computational Complexity

The cost of an algorithm involves both arithmetic, i.e., the number of flops, and communication, i.e., data movement either between different levels of a memory hierarchy or between processors [36]. On multicore and accelerator-based computers, for a data matrix stored externally, the communication cost becomes substantially more expensive compared to the arithmetic [36, 45]. The randomized algorithms operate on a compressed version of the data matrix rather than a matrix itself and therefore can be organized to exploit modern computational environments better than their classical counterparts.

To decompose $\bf A$, the simple version of CoR-UTV incurs the following costs: Step 1 costs $n\ell$, Step 2 costs $2mn\ell$, Step 3 costs $2mn\ell$, Step 4 costs $2m\ell^{2}+2n\ell^{2}$, Step 5 costs $m\ell^{2}+2mn\ell$ (if $\bf D$ is approximated by ${\bf D}_{\text{approx}}$, this step costs $2m\ell^{2}+2n\ell^{2}+3\ell^{3}$), Step 6 costs $8/3\ell^{3}$, Step 7 costs $2m\ell^{2}+2n\ell$. The dominant cost of Steps 1-7 occurs when multiplying $\bf A$ and ${\bf A}^{T}$ with the corresponding matrices. Thus

[TABLE]

The sample size parameter $\ell$ is typically close to the minimal rank $k$. The simple form of CoR-UTV requires either three or two passes (when $\bf D$ is approximated by ${\bf D}_{\text{approx}}$) through data to factor $\bf A$. When the power method is incorporated, CoR-UTV requires either $(2q+3)$ or $(2q+2)$ passes (when $\bf D$ is approximated by ${\bf D}_{\text{approx}}$) over the data with arithmetic costs of $(2q+3)C_{\text{CoR-UTV}}$ or $(2q+2)C_{\text{CoR-UTV}}$, respectively.

In addition to matrix-matrix multiplications and QR decompositions, CoR-UTV performs one QRCP on an $\ell\times\ell$ matrix, however TSR-SVD and SOR-SVD perform an SVD on the $\ell\times\ell$ matrix. The SVD is more expensive than QRCP and, furthermore, recently developed QRCP algorithms based on randomization can perform the factorization with minimum communication costs [37, 38, 46], while standard techniques to compute an SVD are challenging for parallelization [32, 33]. Hence for large matrices to be factored on high performance computing architectures, where the compressed $\ell\times\ell$ matrix does not fit into fast memory, the execution time to compute CoR-UTV can be substantially less than those of TSR-SVD and SOR-SVD. This is an advantage of CoR-UTV over TSR-SVD and SOR-SVD.

4 Robust PCA with CoR-UTV

This section describes how to solve the robust PCA problem using the proposed CoR-UTV method. Robust PCA [39, 40] represents an input low-rank matrix ${\bf M}\in\mathbb{R}^{m\times n}$ whose fraction of entries being corrupted, as a linear superposition of a low-rank matrix ${\bf L}$ and a sparse matrix of outliers ${\bf S}$ such as ${\bf M=L+S}$, by solving the following convex program:

[TABLE]

where ${\|\mbox{\bf B}\|_{*}}\triangleq\sum_{i}\sigma_{i}(\mbox{\bf B})$ is the nuclear norm of any matrix B, ${\|\mbox{\bf B}\|_{1}}\triangleq\sum_{ij}|\mbox{\bf B}_{ij}|$ is the $\ell_{1}$-norm of B, and $\lambda>0$ is a tuning parameter. One efficient method to solve (14) is the method of augmented Lagrange multipliers (ALM) [47]. The ALM method yields the optimal solution, however its bottleneck is computing the costly SVD at each iteration to approximate the low-rank component $\bf L$ of $\bf M$ [40, 48]. To address this issue and to speed up the convergence of the ALM method, the work in [48] proposes a few techniques including predicting the principal singular space dimension, a continuation technique [49], and a truncated SVD by using PROPACK package [50]. The modified algorithm [48] substantially improves the convergence speed, however the truncated SVD [50] employed uses the Lanczos algorithm that (i) is unstable, and (ii) due to the limited data reuse in its operations, has very poor performance on modern architectures [41, 28, 32, 33].

To address this issue, by considering the original objective function proposed in [39, 40, 48], we apply CoR-UTV as a surrogate to the truncated SVD to solve the robust PCA problem. We adopt the continuation technique [49, 48], which increases $\mu$ in each iteration. The proposed ALM-CoRUTV method is given in Alg. 2.

In Alg. 2, for any matrix $\bf B$ with a CoR-UTV decomposition described in Section 3, $\mathcal{C}_{\delta}({\bf B})$ refers to a CoR-UTV thresholding operator defined as:

[TABLE]

where $r$ is the number of diagonals of $\bf T$ greater than $\delta$, $\mathcal{D}_{\delta}({\bf B})$ refers to a singular value thresholding operator defined as $\mathcal{D}_{\delta}({\bf B})={\bf U}_{\text{B}}\mathcal{S}_{\delta}({\bf\Sigma}_{\text{B}}){\bf V}_{\text{B}}^{T}$, where $\mathcal{S}_{\delta}(x)={\text{sgn}(x)\text{max}}(|x|-\delta,0)$ is a shrinkage operator [51], $\lambda$, $\mu_{0}$, ${\bar{\mu}}$, $\rho$, ${\bf Y}_{0}$, and ${\bf S}_{0}$ are initial values. The main operation of ALM-CoRUTV is computing CoR-UTV in each iteration, which is efficient in terms of flops, $O(mnk)$, and can be computed with minimum communication costs.

5 Numerical Experiments

In this section, we present simulations that evaluate the performance of CoR-UTV for approximating a low-rank input matrix. We show that CoR-UTV provides highly accurate singular values and low-rank approximations, and compare CoR-UTV against competing algorithms from the literature. We also employ CoR-UTV for solving the robust PCA problem.

5.1 Rank-Revealing Property & Singular Values Estimation

We first show that CoR-UTV (i) is rank revealer, i.e., the gap in the singular value spectrum of the matrix is revealed, and (ii) provides highly accurate singular values. For the randomized algorithms considered, namely CoR-UTV, TSR-SVD, and SOR-SVD, the results presented are averaged over 20 trials. Each trial was run with the same input matrix with an independent draw of the test matrix. Due to space constraints, we only consider one class of low-rank matrices, and for simplicity we focus on a square matrix.

We construct a noisy rank-$k$ matrix ${\bf A}$ of order $10^{3}$ generated as ${\bf A}={\bf U\Sigma V}^{T}+0.1\sigma_{k}{\bf E}$, where ${\bf U}$ and ${\bf V}$ are random orthonormal matrices, ${\bf\Sigma}$ is diagonal containing the singular values $\sigma_{i}$s that decrease linearly from $1$ to $10^{-9}$, $\sigma_{k+1}=...=\sigma_{10^{3}}=0$, and ${\bf E}$ is a normalized Gaussian matrix. We set $k=20$.

We compare the singular values of the matrix computed by CoR-UTV against those of competing methods such as the SVD [28], QRCP [29], UTV [52] and TSR-SVD [32]. For CoR-UTV and TSR-SVD, we arbitrarily set the sample size parameter to $\ell=2k$. Both algorithms require the same number of passes over $\bf A$, either two or $2q+2$ when the power method is used, to perform a factorization.

The results are shown in Fig. 1. It is observed that (i) CoR-UTV strongly reveals the numerical rank $k$, (ii) with no power iterations ($q=0$), CoR-UTV provides very good approximations to singular values and outperforms TSR-SVD in approximating both leading and trailing singular values, (iii) with $q=2$, CoR-UTV delivers singular values as accurate as the optimal SVD, (iv) QRCP only suggests the gap in the singular spectrum, and gives a fuzzy approximation to singular values of the matrix.

5.2 Low-Rank Approximation

We now compare the low-rank approximation constructed by our method against those of the SVD, QRCP, TSR-SVD, and SOR-SVD [43]. We construct a rank-$k$ approximation ${\hat{\bf A}}_{\text{out}}$ to ${\bf A}$ by varying the sample size parameter $\ell$ with the rank fixed, and calculate the error:

[TABLE]

The results are shown in Fig. 2. It is observed that (i) when $q=0$, CoR-UTV and SOR-SVD show similar performances, while TSR-SVD shows the worst performance, (ii) when $q=2$, the errors resulting from CoR-UTV show no loss of accuracy compared to the optimal SVD. In this case, QRCP has the poorest performance.

5.3 Robust Principal Component Analysis

Here, we examine the efficiency and efficacy of ALM-CoRUTV in Alg. 2 for recovering the low-rank and sparse components of data. We compare the results obtained with those of the efficient inexact ALM method by [48], called InexactALM hereafter.

Robust PCA represents an input low-rank matrix ${\bf M}\in\mathbb{R}^{m\times n}$, whose a fraction of entries being corrupted, as a linear superposition of a low-rank matrix ${\bf L}$ and a sparse matrix of outliers ${\bf S}$ such as ${\bf M=L+S}$, by solving the following convex program:

[TABLE]

where ${\|\mbox{\bf B}\|_{*}}\triangleq\sum_{i}\sigma_{i}(\mbox{\bf B})$ is the nuclear norm of any matrix B, ${\|\mbox{\bf B}\|_{1}}\triangleq\sum_{ij}|\mbox{\bf B}_{ij}|$ is the $\ell_{1}$-norm of B, and $\lambda>0$ is a tuning parameter [53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63], [64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75], [76, 77, 78, 79, 80, 81, 82, 83, 84]. The matrix ${\bf L}$ is generated as ${\bf L}={\bf U}{\bf V}^{T}$, where ${\bf U}$, ${\bf V}\in\mathbb{R}^{n\times k}$ have standard Gaussian distributed entries. The error matrix ${\bf S}$ has $s$ non-zero entries independently drawn from the set $\{$-80, 80$\}$. We apply the ALM-CoRUTV and InexactALM algorithms to $\bf M$ to recover ${\bf L}$ and ${\bf S}$. The numerical results are summarized in Table 1, where the rank of $\bf L$ $r({\bf L})=0.05\times n$ and $s=\|{\bf S}\|_{0}=0.05\times n^{2}$.

In our experiments, we adopt the initial values suggested in [48]. The algorithms are terminated when ${\|{\bf M}-{\bf L}^{out}-{\bf S}^{out}\|_{F}}<10^{-5}{\|{\bf M}\|_{F}}$ is satisfied, where $({\bf L}^{out},{\bf S}^{out})$ is the pair of output of either algorithm. In the Table, $Time(s)$ refers to the runtime in seconds, $Iter.$ refers to the number of iterations, and $\zeta={\|{\bf M}-{\bf L}^{out}-{\bf S}^{out}\|_{F}}/{{\|{\bf M}\|_{F}}}$ refers to the relative error.

CoR-UTV requires a prespecified rank $\ell$ to perform the factorization. Thus, we set $\ell=2k$, as a random start, and $q=1$. The results in Table 1 show that ALM-CoRUTV detects the exact rank $k$ of the input matrix, provides the exact optimal solution, and outperforms InexactALM in terms of runtime.

6 Conclusions

In this paper, we have presented CoR-UTV for computing low-rank approximations of an input matrix. Simulations show that CoR-UTV reveals the numerical rank more sharply than QRCP, and provides results as good as those of the optimal SVD. CoR-UTV is more efficient than SVD, QRCP, UTV, TSR-SVD and SOR-SVD in terms of cost. CoR-UTV can better exploit advanced computational platforms by leveraging higher levels of parallelism than all compared algorithms. We have also applied CoR-UTV to solve the robust PCA problem via the ALM method. Simulations show that ALM-CoRUTV outperforms efficiently implemented InexactALM.