Quantum-enhanced least-square support vector machine: simplified quantum algorithm and sparse solutions

TL;DR

This paper introduces a simplified quantum algorithm for LS-SVM that leverages continuous variables and hybrid quantum-classical methods to achieve exponential speed-up and sparse solutions, enhancing machine learning capabilities.

Contribution

It presents a novel quantum algorithm using continuous variables for matrix inversion and a hybrid approach for sparse LS-SVM solutions, integrating kernel methods for improved performance.

Findings

Achieves exponential speed-up in matrix inversion for LS-SVM

Enables extraction of smaller datasets from large quantum-encoded data

Incorporates kernel methods leveraging quantum properties

Abstract

Quantum algorithms can enhance machine learning in different aspects. Here, we study quantum-enhanced least-square support vector machine (LS-SVM). Firstly, a novel quantum algorithm that uses continuous variable to assist matrix inversion is introduced to simplify the algorithm for quantum LS-SVM, while retaining exponential speed-up. Secondly, we propose a hybrid quantum-classical version for sparse solutions of LS-SVM. By encoding a large dataset into a quantum state, a much smaller transformed dataset can be extracted using quantum matrix toolbox, which is further processed in classical SVM. We also incorporate kernel methods into the above quantum algorithms, which uses both exponential growth Hilbert space of qubits and infinite dimensionality of continuous variable for quantum feature maps. The quantum LS-SVM exploits quantum properties to explore important themes for SVM such as sparsity and kernel methods, and stresses its quantum advantages ranging from speed-up to the potential capacity to solve classically difficult machine learning tasks.

Equations

$${\min }_{w,b}\frac{1}{2}\Vert w{\Vert }^2+\frac{1}{2}\gamma \sum _{i=1}^M{\xi }_i$$

$$s.t.y_i(w^T\varphi (x_i)$$

$$\{\begin{array}{ll}& {\alpha }_i(f(x_i)y_i-1+{\xi }_i)=0,\\ & 1-{\xi }_i-f(x_i)y_i\le 0,\\ & {\alpha }_i\ge 0.\end{array}$$

$${\min }_{w,b}\frac{1}{2}\Vert w{\Vert }^2+\frac{1}{2}\gamma \sum _{i=1}^M{\xi }_i$$

$$s.t.y_i(w^T\varphi (x_i)+b)=1-{\xi }_i,{\xi }_i\ge 0.$$

$$\left[\begin{array}[]{c}\alpha\\ b\end{array}\right]=\left[\begin{array}[]{cc}K+\gamma^{-1}I_{MM}&\mathbf{1}^{T}\\ \mathbf{1}&0\\ \end{array}\right]^{-1}\left[\begin{array}[]{c}y\\ 0\end{array}\right]=M_{s}^{-1}y_{s}.$$

$${\min }_{\alpha ,b}\frac{1}{2}\alpha ({\gamma }^{-1}K+K{K}^T){\alpha }^T+\alpha K(1^{T}b-y)-y^T{1}^{T}b+\frac{M{b}^2}{2}.$$

$$({\gamma }^{-1}K+K{K}^T$$

$$b=\frac{1}{M}(1y-1K{\alpha }^T).$$

$$M_s^{-1}=\frac{i}{\sqrt{2\pi }}{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }\psi (p_1)p_2{e}^{-\frac{p_2^2}{2}}{e}^{i{M}_s{p}_1{p}_2}d{p}_{1}d{p}_2.$$

$$\psi(p_{1})=\{\begin{array}[]{cc}1,&0<p_{1}\\ 0,&otherwise\\ \end{array}.$$

$$|q⟩$$

$$|g⟩$$

$$\psi(p_{1})=\{\begin{array}[]{cc}1,&0<p_{1}<L\\ 0,&otherwise\\ \end{array}.$$

$$f(\hat{x})$$

$$\displaystyle=sgn((\phi(\hat{x}),1)\left[\begin{array}[]{cc}\phi(A)^{T}&0\\ 0&1\\ \end{array}\right]\left[\begin{array}[]{c}\alpha\\ b\end{array}\right])$$

$$=sgn(x_s{A}_s{\alpha }_s).$$

$$G_{1}=\left[\begin{array}[]{cc}K&0\\ 0&0\\ \end{array}\right]G_{2}=\left[\begin{array}[]{cc}\gamma^{-1}I&0\\ 0&0\\ \end{array}\right]G_{3}=\left[\begin{array}[]{cc}0&\mathbf{1}^{T}\\ \mathbf{1}&0\\ \end{array}\right].$$

$$|{\psi }_2⟩=\sum _{k=1}^{M+1}{c}_k|{\hat{u}}_k⟩e^{i{\hat{\lambda }}_k\widehat{p_1}\widehat{p_2}}|q⟩|g⟩.$$

$$|{\psi }_3⟩=\sum _{k=1}^{M+1}{c}_k{\lambda }_k^{-1}|{\hat{u}}_k⟩.$$

$$f_{\Omega }(\hat{x})=sgn(\varphi (\hat{x})\varphi (A{)}_{\Omega }^T{\alpha }_{\Omega }^T+b^{\prime })$$

$$({\gamma }^{-1}{K}_{{\Omega }_R}$$

$$=K_{{\Omega }_R}(y_{{\Omega }_R}^T-\frac{1y^T}{M}{1}_{{\Omega }_R}^T),$$

$$b=\frac{1}{M}(1y^T-1_{{\Omega }_R}{K}_{{\Omega }_R}{\alpha }_{{\Omega }_R}^T).$$

$$|{\psi }_2^{{}^{\prime }}⟩=\sum _{k=1}^R{\sigma }_k|u_k⟩|v_k⟩|\eta {\lambda }_k{⟩}_{q_3},$$

$$|{\beta }_1⟩=\frac{1}{\sqrt{2}}(|0⟩|u_k⟩+|1⟩|{\psi }_1⟩),$$

$$|{\beta }_2⟩=\frac{1}{\sqrt{2}}(|0⟩|u_k⟩+|1⟩|{\psi }_y⟩),$$

$$|{\beta }_3⟩=\frac{1}{\sqrt{2}}(|0⟩|v_k⟩+|1⟩|{\psi }_{\varphi (\hat{x})}⟩).$$

$$K(x,z)$$

$$=\varphi (x)\varphi (z{)}^T.$$

$$K(x,z)=⟨\varphi (x)|\varphi (z)⟩=⟨x^{\prime }|z^{\prime }{⟩}^d.$$

$${\lim }_{k\to \infty }(1+\frac{s}{k}{)}^k=e^s.$$

$$|\varphi (x^{\prime })⟩=\frac{1}{\sqrt{N_{x^{\prime }}}}\sum _{i_1,\cdots ,i_k=1}^{M+1}\frac{x_{i_1}\cdots x_{i_k}}{(\omega \sqrt{k}{)}^{-k}}|i_1⟩\cdots |i_k⟩.$$

$$\varphi_{A}=\left[\begin{array}[]{cccc}x_{1}^{\prime}&0&\cdots&0\\ 0&x_{2}^{\prime}&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&x_{M}^{\prime}\\ \end{array}\right]\in R^{M\times(M+1)^{2}},$$

$${\phi }_A$$

$$=I_{M\times M}{\Sigma }_{{\phi }_A}{F}^{†}=\sum _{i=1}^M\sqrt{x_i^{\prime }(x_i^{\prime }{)}^T}{e}_i{f}_i^{†}.$$

Full text

††preprint: APS/123-QED

I Introduction

Supervised learning aims to establish a mapping between features and labels of data by learning from a dataset Qiu et al. (2016); Dunjko andB̃riegel (2018); Novaković (2016); Hsu andL̃in (2002). The learned mapping is further used to classify new data. A support vector machine (SVM) serves as a classifier by learning a hyperplane that separates two classes of data in a feature space, where the feature map can be implicitly generated with kernel methods Wu andZ̃hou (2005); Suykens andṼandewalle (1999); Schölkopf et al. (1999); Sánchez A (2003). As a result of spare solutions, the number of samples involved for inference can be small. Due to the flexibility of incorporating kernels and the sparsity of solutions, SVM is powerful and efficient in classifying complicated data and thus has wide real-world applications.

Recently, the frontier of machine learning is pushed forward with quantum computing Gu et al. (2012); Sarma et al. (2019); Wiebe et al. (2012); Dunjko et al. (2016); Lloyd et al. (2016); Huang et al. (2017); Lloyd andW̃eedbrook (2018), which utilizes the capacity of quantum computing to manipulate data in a large Hilbert space. By exploiting quantum properties such as superposition and entanglement, machine learning can be enhanced in several aspects, such as quantum speed-up, quantum-enhanced feature maps, better generalization, and so on Zhang et al. (2019a); Bremner et al. (2016); Douce et al. (2017).

As for the SVM, a quantum algorithm with exponential speed-up has been proposed Rebentrost et al. (2014) and verified with a proof-of-principle experiment Li et al. (2015). Kernel methods has also been discussed, where feature map is done explicitly by encoding classical data as quantum states Benedetti et al. (2019); Schuld andK̃illoran (2019); Havlíček et al. (2019). However, the quantum algorithm is designed for a least-square version of SVM, which is lack of sparsity. Also, quantum matrix inversion Harrow et al. (2009); Rebentrost et al. (2018), a main ingredient of the algorithm, involves many ancillary qubits and complicated circuits for eigenvalue/singular-value inversion. Stressing those issues can help realize feasible quantum algorithm for support vector machine with less quantum resources in near-term quantum technology.

In this paper, we study quantum-enhanced least-square support vector machine, and propose a simplified quantum algorithm as well as sparse solutions. Remarkably, those quantum algorithms are designed deliberately so as to be applicable when considering quantum feature maps. Firstly, we propose a quantum algorithm that assists the matrix inversion with two continuous variables Lloyd (2003); Lau et al. (2017); Zhang et al. (2019b); Arrazola et al. (2018), which greatly simplifies the algorithm for quantum LS-SVM and reduces the use of ancillary qubits. Secondly, we give sparse solutions for quantum LS-SVM. The large classical dataset is firstly encoded into a quantum state, and then is compressed into a much smaller dataset, while essential information for classification task is kept. This procedure is conducted with quantum matrix toolbox, including quantum principal component analysis (QPCA) Lloyd et al. (2014) and quantum singular-value threshold algorithm (QSVT) Duan et al. (2017, 2018). The transformed dataset is further processed using classical LS-SVM. In this regard, it is a hybrid quantum-classical LS-SVM. The quantum enhancement manifests in a manipulation of data with speed-up with quantum matrix toolbox, as well as quantum feature maps that may be classically intractable Bremner et al. (2016); Douce et al. (2017); Schuld andK̃illoran (2019); Havlíček et al. (2019); Harrow et al. (2009). Our investigation of quantum-enhanced LS-SVM thus contributes to important topics in SVM with feasible quantum algorithms that exploits quantum advantages.

The remainder of the paper is organized as follows: We introduce an improved quantum hybrid variables method, give a brief overview of the LS-SVM algorithm in Sec. II, put forward a quantum LS-SVM with hybrid variables (HVQ-SVM) algorithm and propose a hybrid variables algorithm for sparse solution of LS-SVM model (QSLS-SVM) in Sec. III, discuss the regularization problem and advantages of algorithm in Sec. IV, present our conclusions in Sec. V.

II The review of SVM, LS-SVM and an improved quantum hybrid variables method

In this section, we firstly introduce the general notations and the SVM, LS-SVM model Wu andZ̃hou (2005); Suykens andṼandewalle (1999). Next, we present an improved quantum hybrid variables method based on the Ref.Zhang et al. (2019b); Arrazola et al. (2018).

II.1 Notations

The each vector $x$ can be defined as $y=(y_{1},y_{2},\cdots,y_{M}),y\in R^{1\times N}$. A set of training data is defined as $\{x_{i},y_{i}\},i=1,2,\cdots,M$, where $x_{i}\in R^{1\times N}$ denote a vector with $N$ features and $y_{i}\in\{-1,1\}$ is label corresponding to $x_{i}$. Let $\mathbf{1}=(1,1,\cdots,1)$ and $y=(y_{1},y_{2},\cdots,y_{M})$. Define $\phi$ as the mapping function, the original input space can be mapped into the feature space and the inner product $x_{i}x_{j}^{T}$ can be represented as $\phi(x_{i})\phi(x_{j})^{T}$ in this feature space. Let $I_{M\times M}$ represents the $M\times M$ identity matrix and $e_{i}$ denotes the i-th row of $I_{M\times M}$. Define the matrix of containing original data $A=(x_{1}^{T},x_{2}^{T},\cdots,x_{M}^{T})^{T}$ and $\phi(A)=(\phi(x_{1}^{T}),\phi(x_{2}^{T}),\cdots,\phi(x_{M}^{T})^{T})$. Let the kernel matrix $K$ can be $K_{ij}=K(x_{i},x_{j})=\phi(x_{i})\phi(x_{j})^{T}$. The singular value decomposition (SVD) of matrix $A$ is $A=\sum_{i=1}^{R}\sigma_{i}u_{i}v_{i}^{T}$ where $\sigma_{i}$ and $u_{i},v_{i}$ are eigenvalues and the corresponding eigenvectors respectively. The SVD of $\phi(A)$ can be denoted as $\phi(A)=U_{M\times M}\Sigma_{M\times N}V_{N\times N}^{T}=U_{M\times R}\Sigma_{R\times R}V_{N\times R}^{T}$ where $U_{M\times M}=(U_{M\times R},U_{M\times(M-R)})=(u_{1},u_{2},\cdots,u_{M})$ and $V_{N\times N}^{T}$ has similar expression. The generalized inverse of $\phi(A)$ is $\phi(A)^{{\dagger}}=V_{R\times N}\Sigma_{R\times R}^{-1}U_{M\times R}^{T}$. The distance between the two vectors $x_{i},x_{j}$ can be $|x_{i}-x_{j}|=\sqrt{\sum_{k=1}^{M}(x_{ik}-x_{jk})^{2}}$.

II.2 The SVM and LS-SVM models

For the large-scale datasets $\{x_{i},y_{i}\},i=1,2,\cdots,M$, the goal of SVM or LS-SVM is to output parameters $w\in R^{N_{\phi}\times 1}$ and $b\in R$ where the value of $N_{\phi}$ can be related to $\phi$. The new data $\hat{x}$ can be distinguished by the model containing these parameters: $f(\hat{x})=sgn(w^{T}\phi(\hat{x})^{T}+b)$.

The soft margin SVM model finds these optimal parameters $w,b$ by solving the following optimization task:

[TABLE]

where $\gamma$ is a penalty value, $\xi_{i},i=1,\cdots,M$ are the slack variables. By solving the dual problem of Eqs.(1) with Lagrangian multipliers $\alpha_{i},i=1,2,\cdots,M$ where $\alpha=(\alpha_{1},\alpha_{2},\cdots,\alpha_{M})$, parameters $w$ can be represented as $w=\sum_{i=1}^{M}\alpha_{i}x_{i}$. This SVM model satisfies the related KKT condition

[TABLE]

In this KKT condition, there are $f(x_{i})y_{i}=1-\xi_{i}$ and it shows that $x_{i}$ is a support vector if the parameter $\alpha_{i}\neq 0$ . The parameter $\alpha_{i}$ has no effect on $f(x_{i})$ when $\alpha_{i}=0$. The SVM model has sparsity because of these parameters $\alpha_{i}=0$.

The LS-SVM model uses equality instead of inequality in terms of constraint condition and finally the quadratic programming (QP) problem can be transformed to solve matrix inversion problem.

[TABLE]

where this equality can be replaced as $(w^{T}\phi(x_{i})+b)=y_{i}-y_{i}\xi_{i}$. The Eqs.(3) can be used to construct Lagrange function with Lagrangian multipliers $\alpha_{i}$ and other parameters. By taking partial derivatives of this function and eliminating other parameters, $\alpha,b$ satisfy $w^{T}=\sum_{i=1}^{M}\alpha_{i}\phi(x_{i})^{T}$ and have the following formulation

[TABLE]

The original LS-SVM model corresponding to Eqs.(4) is inappropriate for calculating a single entry of vectors and we employ the equivalent model (PLS-SVM) Zhou (2015) instead of the original one. PLS-SVM can be obtained via the representer theorem Schölkopf et al. (2001) and the corresponding optimization problem can be described as follows:

[TABLE]

The Eqs.(4) can be changed to the following formulation:

[TABLE]

II.3 Quantum matrix inversion assisted with continuous variables

Based on the Ref.Zhang et al. (2019b); Arrazola et al. (2018), we introduce an improved quantum hybrid variables method as this operation on matrix inversion. Compared with original method [32], we can simply the algorithm. Moreover, the width $L$ can be employed to control size of eigenvalues. Assume the SVD composition of $M_{s}$ is $M_{s}=\sum_{i=1}^{M+1}\hat{\lambda}_{i}\hat{u}_{i}\hat{u^{{\dagger}}}_{i}^{T}$. The inverse matrix $M_{s}^{-1}$ has following integral form:

[TABLE]

where

[TABLE]

Then our goal is to use continuous quantum variables to perform matrix inversion. We use CV quantum states

[TABLE]

where the step function state of Eqs.(7) is unphysical and the finite width $L$ can be used to approximate ideal step function

[TABLE]

The quantum form of the operation $M_{s}^{-1}u_{i}$ is equal to this process: constructing the unitary operation $e^{iM_{s}\hat{p_{1}}\hat{p_{2}}}$ [14,26] and performing this operation on $|\hat{u}_{i}\rangle|q\rangle|g\rangle$, we approximately obtain the $\hat{\lambda}_{i}^{-1}|\hat{u}_{i}\rangle$ with a certain probability after using homodyne dectection.

III HVQ-SVM and QSLS-SVM algorithms

In this section we design LS-SVM algorithm of quantum version and this algorithm consists of HVQ-SVM and QSLS-SVM algorithms. The HVQ-SVM algorithm is a quantum algorithm in overall procedure and can be better than all-qubits version. The QSLS-SVM algorithm can be employed to obtain a classical solution with equivalent parameters and consists of quantum part and classical part.

III.1 The HVQ-SVM Algorithm

We first transform this LS-SVM model to a quantum problem. The kernel classification function of LS-SVM model have the following form:

[TABLE]

Then the vectors $x_{s}$ and $y_{s}$ are encoded as quantum states $|\psi_{x_{s}}\rangle$ and $|\psi_{y_{s}}\rangle$ respectively via QRAM Giovannetti et al. (2008a, b). And the matrices $A_{s}$ can be loaded as $|\psi_{A_{s}}\rangle$. Thus, the classification problem can be inner product of the two quantum states $|\psi_{A_{s}}\rangle$ and $|\psi_{x_{s}}\rangle M_{s}^{-1}|\psi_{y_{s}}\rangle$. The core of this problem is to obtain $M_{s}^{-1}|\psi_{y_{s}}\rangle$. Specifically, the state $|\psi_{y_{s}}\rangle$ has the form $|\psi_{y_{s}}\rangle=\sum_{i=1}^{M+1}y_{i}|i\rangle=\sum_{i=1}^{M+1}c_{i}|\hat{u}_{i}\rangle$ where $y_{M+1}=0$ and the latter is a representation of $|\psi_{y_{s}}\rangle$ on this basis $\{|\hat{u}_{i}\rangle\}$. And $|\psi_{x_{s}}\rangle,|\psi_{A_{s}}\rangle$ have similar expression when we deal with a linear LS-SVM model. Here we take polynomial kernel function $K(x,z)=(\langle x,z\rangle+1)^{d}$ as a example to describe the preparation of $|\psi_{x_{s}}\rangle,|\psi_{A_{s}}\rangle$ when we handle the non-linear LS-SVM model. Assume $\hat{x}^{\prime}=(\hat{x},1,1),x_{i}^{\prime}=(x_{i},1,0)^{T},$ and $x_{M+1}^{\prime}=(0,0,\cdots,0,1)^{T}$, these vectors are encoded as quantum states $|\hat{x}^{\prime}\rangle,|x_{i}^{\prime}\rangle,i=1,\cdots,M+1$ and $K(x_{i},x_{j})=\langle x_{i}^{\prime}|x_{j}^{\prime}\rangle^{d}$. Then the states $|\psi_{x_{s}}\rangle,|\psi_{A_{s}}\rangle$ can be denoted as $|\psi_{x_{s}}\rangle=\underbrace{|\hat{x}^{\prime}\rangle\otimes\cdots\otimes|\hat{x}^{\prime}\rangle}_{d}$ and $|\psi_{A_{s}}\rangle=\sum_{i=1}^{M+1}|i\rangle|\phi(x_{i}^{\prime})\rangle$ where $|\phi(x_{i}^{\prime})\rangle=\underbrace{|x_{i}^{\prime}\rangle\otimes\cdots\otimes|x_{i}^{\prime}\rangle}_{d}$. The detail description can be found in appendix A.

Here we use the hybrid variables method in our algorithm. However, matrix $M_{s}$ is full rank and unitary operation $e^{iM_{s}\hat{p_{1}}\hat{p_{2}}}$ can not be constructed. Matrix inversion can be constructed as a combination of unitary operators Rebentrost et al. (2014). Therefore, $e^{iM_{s}\hat{p_{1}}\hat{p_{2}}}=\prod_{k=1}^{3}e^{iG_{k}\hat{p_{1}}\hat{p_{2}}}$ where

[TABLE]

The whole process of HVQ-SVM Algorithm is described as follows:

1: Initialization. The discrete quantum state is encoded as $|\psi_{y_{s}}\rangle$. Two-mode is initialized in $|q\rangle|g\rangle$. Then the initial quantum state is prepared as $|\psi_{1}\rangle=|\psi_{y_{s}}\rangle|q\rangle|g\rangle$.

2: Phase estimation. Applying the uniform operation $e^{iM_{s}\hat{p_{1}}\hat{p_{2}}}$ on $|\psi_{1}\rangle$ which leads to

[TABLE]

3: Homodyne detection. The operation $I\otimes|0_{p_{1}}\rangle\langle 0_{p_{1}}|\otimes|0_{p_{2}}\rangle\langle 0_{p_{2}}|$ can be applied on $|\psi_{2}\rangle$ and two-mode can be eliminated to obtain:

[TABLE]

where $\langle q|0_{p_{1}}\rangle=\langle r|0_{p_{2}}\rangle=1$. It is equal to perform a homodyne measurement on two-mode and post-select the result $p=0$.

4: Measurement. Assume $|\psi_{4}\rangle=|\psi_{x_{s}}\rangle|\psi_{3}\rangle$, an ancillary qubit can be used to construct an entangled state $|\psi_{5}\rangle=\frac{1}{\sqrt{2}}(|0\rangle|\psi_{4}\rangle+|1\rangle|\psi_{A_{s}}\rangle)$. Performing a $\sigma_{x}$ measurement and the inner product $\langle A_{s}|\psi_{4}\rangle$ can be obtained.

We omit the normalization factor in overall procedure of this algorithm and choose the suitable width $L$ to preserve the fidelity and reduce the error. Notably this error can be taken as a regulation method (see the appendix B).

Chapter \thechapter Analysis.

We briefly discuss the time complexity of our algorithm. The main part of complexity is the preparation of initial quantum state and construction of unitary transformation $e^{iM_{s}\hat{p_{1}}\hat{p_{2}}}$ or $e^{iG_{k}\hat{p_{1}}\hat{p_{2}}},k=1,2,3$. The initial quantum state can be prepared via qRAM and costs $O(logMN)$. In the quantum phase estimation, the unitary operation $e^{iM_{s}\hat{p_{1}}\hat{p_{2}}}$ can be constructed by this method outlined in Ref.Lau et al. (2017); Zhang et al. (2019b). It costs time $O(\varepsilon^{-1}log(MN))$ where $\varepsilon^{-1}$ denotes precision of generating the eigenvalues. The total evolution time complexity of our algorithm is $O(\varepsilon^{-1}log(MN))$.

III.2 The QSLS-SVM algorithm

There are a series of algorithms Suykens et al. (2000); ling Chen et al. (2014); Mall andS̃uykens (2015); Silva et al. (2015); Zhou andL̃iu (2017); Chen andZ̃hou (2018) to solve sparsity of LS-SVM. The sparsity can reflect advantage that we can use small samples to reveal the primary performance of entire datasets. Another advantage is that these important samples can be obtained by using quantum algorithm in $O(log[M])$ time. Notably, the quantum algorithm can be used as a subroutine for overall process of algorithm since we obtain classical information by measuring the output result of quantum algorithm.

As a better way, the P-LSSVM model can be constructed to solve sparse problem. The method about ny$\ddot{o}$trom approximation Fowlkes et al. (2004); Li et al. (2014) outlined in Ref.Suykens et al. (2000) can be used to solve this problem. However, the cost of finding a full-rank submatrix of $K$ is expensive. Here we employ the optimal matrix approximation instead of above method to achieve the sparse solution of P-LSSVM.

Define the left eigenvector matrix of $\phi(A)$ as $\Omega=U_{M\times M}\in R^{M\times M}$, we show that $\phi(A)_{\Omega}$ and $\phi(A)$ have same result in classification function where $\phi(A)_{\Omega}=\Omega^{T}\phi(A)$. Assume $\alpha_{\Omega}^{T}=\Omega^{T}\alpha^{T},y_{\Omega}^{T}=\Omega^{T}y^{T},\mathbf{1}_{\Omega}^{T}=\Omega^{T}\mathbf{1}^{T}$, $K_{\Omega}=\phi(A)_{\Omega}\phi(A)_{\Omega}^{T}$ and the classification function has the following form:

[TABLE]

Compared with the Eqs.(9), we have $b=b^{\prime}$ and $f_{\phi}(\hat{x})=f(\hat{x})$. Let’s take it a step further. In Eqs.(13), the inner product $\phi(A)_{\Omega}^{T}\alpha_{\Omega}^{T}$ can be expressed as follows: $\phi(A)_{\Omega}^{T}(\alpha_{\Omega}^{{}^{\prime}})^{T}$ where the sparse vector $\alpha_{\Omega}^{{}^{\prime}}=(\alpha_{\Omega_{R}},0)$ and vector $\alpha_{\Omega_{R}}$ represents the top-R entries of $\alpha_{\Omega}$. Thus, $\alpha_{\Omega_{R}}$ and $b$ are the valid parameter that we get.

In the same way, we denote the top-R entries of $\mathbf{1}_{\Omega},y_{\Omega}$ as $\mathbf{1}_{\Omega_{R}},y_{\Omega_{R}}$, respectively. And suppose the submatrix of $K_{\Omega}$ as $K_{\Omega_{R}}=\Sigma_{R\times R}$, the solution Eqs.(6) of P-LSSVM can be simplified as:

[TABLE]

From this formulation, we come to the conclusion that the product $\mathbf{1}y^{T}$ and the vectors $\mathbf{1}_{\Omega_{R}},K_{\Omega_{R}},y_{\Omega_{R}}$ are the core of target parameters $\alpha_{\Omega_{R}},b$. Firstly, we encode the vector $y^{T}$ as a quantum state $|\psi_{y}\rangle=\sum_{i=1}^{M}y_{i}|i\rangle$ and apply Hadamard gate $H^{\otimes^{\lceil logM\rceil+1}}$ on $|0\rangle$ to obtain uniform superposition state $|\psi_{\mathbf{1}}\rangle=\sum_{i=1}^{M}|i\rangle$. The value $\mathbf{1}y^{T}$ can be calculated by $\langle\psi_{y}|\psi_{\mathbf{1}}\rangle$. Next, we use the quantum toolbox such as QPCA or QSVT Lloyd et al. (2014); Duan et al. (2018) to extract singular values $\sigma_{i}$ and eigenvectors $u_{i},v_{i}$. These singular vectors can be used to calculate $\mathbf{1}_{\Omega_{R}},y_{\Omega_{R}}$ via the product $u_{i}^{T}y^{T},u_{i}^{T}\mathbf{1}^{T}$ in quantum form. In addition, to avoid the complex classical calculation caused by large-scale features, e.g. $N=O(M)$, we need to obtain $\phi(\hat{x})\phi(A)_{\Omega}^{T}$ of Eqs.(13) by computing the product $\sigma_{i}\phi(\hat{x})v_{i}$.

Based on the above analysis, we divide QSLS-SVM into quantum and classical subroutines. In the quantum part of the algorithm, the goal we want to achieve is to gain the $\mathbf{1}_{\Omega_{R}},K_{\Omega_{R}},y_{\Omega_{R}}$ and $\phi(\hat{x})\phi(A)_{\Omega}^{T}$. And these information can be employed to solve the Eqs.(13) and Eqs.(14) via classical calculation.

There are a couple of other things to notice before we go into the algorithm. The matrix $\phi(A)$ can be loaded as the quantum state $|\psi_{\phi(A)}\rangle=\sum_{i=1}^{M}\sum_{j=1}^{N_{\phi}}\phi(A)_{ij}|i\rangle|j\rangle=\sum_{i=1}^{R}\sigma_{i}|u_{i}\rangle|v_{i}\rangle$ via qRAM Giovannetti et al. (2008a, b) and Gram-Schmidt decomposition. And the multiple unitary operations $e^{i\eta K\hat{p}_{3}}$ Lau et al. (2017); Zhang et al. (2019b) can be constructed in quantum phase estimation of this algorithm. In this paper, we take QPCA algorithm as the tool, which is based on a single continuous variable Lloyd andB̃raunstein (1999); Lau et al. (2017) corresponding to orthogonal conjugate operator $\hat{p}_{3},\hat{q}_{3}$, i.e. $[\hat{p}_{3},\hat{q}_{3}]=i$ and $|q_{3}\rangle_{q_{3}}=\frac{i}{\sqrt{2\pi}}\int dp_{3}e^{-iq_{3}p_{3}}|p_{3}\rangle_{p_{3}}$. The algorithm process is described as follows:

1:State preparation. The training data $\{x_{i}\}$ can be encoded as the quantum state $|\psi_{\phi(A)}\rangle=\sum_{i=1}^{M}\sum_{j=1}^{N_{\phi}}\phi(A)_{ij}|i\rangle|j\rangle$ with kernel map $\phi$. And one mode can be initialized in $|0\rangle_{q_{3}}$. The initial state can be set to $|\psi_{1}^{{}^{\prime}}\rangle=|\psi_{\phi(A)}\rangle|0\rangle_{q_{3}}$.

2:Phase estimation. Perform the operation $e^{i\eta K\hat{p}_{3}}$ on $|\psi_{1}^{{}^{\prime}}\rangle$ which can lead to

[TABLE]

where $\lambda_{k}=\sigma_{k}^{2}$.

3:Homodyne detection. After a homodyne detection on mode $q_{3}$, the state $|\psi_{2}^{{}^{\prime}}\rangle$ collapses to $|\psi_{3}^{{}^{\prime}}\rangle=|u_{k}\rangle|v_{k}\rangle$ and we obtain the eigenvalue $\lambda_{k}$.

4:Numerical calculation. Vectors $y,\mathbf{1}$ and new data $\hat{x}$ can be loaded as quantum states $|\psi_{y}\rangle,|\psi_{\mathbf{1}}\rangle,|\psi_{\phi(\hat{x})}\rangle$, respectively. For the k-th entries of vectors $\mathbf{1}_{\Omega_{R}},y_{\Omega_{R}}$ and $\phi(\hat{x})\phi(A)_{\Omega}^{T}$, an ancillary qubit can be used to construct entangled states

[TABLE]

Then, perform a $\sigma_{x}$ measurement on the ancillary qubit of the states $|\beta_{j}\rangle,j=1,2,3$ which can obtain the k-th entries of target vectors.

5: Repeat step1 through step4 until we have all the components of the target vectors. An entangled state $|\beta_{4}\rangle=\frac{1}{\sqrt{2}}(|0\rangle|y\rangle+|1\rangle|\psi_{\phi(\hat{x})}\rangle)$ can be constructed to obtain $\mathbf{1}y^{T}$.

6: Classical calculation. These vectors and $\mathbf{1}y^{T}$ can be used to get the $\alpha_{\Omega_{R}}$ and $b$ by solving the Eqs.(14).

Finally, we identify categories which the new data $\hat{x}$ belongs to via classification function $f(\hat{x})$. However, the new data can not be used alone in the above-presumed large-scale features. Taking small-scale features into account, e.g. $N\ll M$, we can construct classical model after obtaining the target parameters $\alpha_{\Omega_{R}}$ and $b$, i.e. suppose $w_{\Omega}=\phi(A)_{\Omega}^{T}(\alpha_{\Omega}^{{}^{\prime}})^{T}$, we have $f(\hat{x})=sgn(w_{\Omega}^{T}\phi(\hat{x})^{T}+b)$. This case is more suitable for linear LS-SVM.

Chapter \thechapter Analysis.

We briefly discuss the time complexity of our algorithm. The main part of complexity is the preparation of initial quantum state and operation $e^{i\eta K\hat{p}_{3}}$. Similar to the HVQ-SVM algorithm, the total evolution time complexity of the QSLS-SVM algorithm is $O(\varepsilon^{-1}log(MN))$ in step1-ster4. And actually, we need to repeat $O(R)$ times to achieve numerical calculation. In addition, we should spend $O(N+R^{3})$ to for the classical calculation. The total time complexity is $O(N+R^{3}+R(\varepsilon^{-1}log(MN)))$.

IV DISCUSSIONS AND CONCLUSIONS

Both HVQ-SVM and QSLS-SVM algorithms have their own advantages and different applications. The former is simplified in algorithm and the latter can get a sparse solution. For the analysis of HVQ-SVM, please refer to appendix B. Here we mainly analyse the QSLS-SVM algorithm in this section. In practice, the cost of obtaining all the eigenvalues by QPCA algorithm is expensive, and we discuss this problem in the following paragraph. In addition, we also analyze the effect of our method on SVM.

In the QSLS-SVM algorithm mentioned above, the requirement for the matrix $\phi(A)$ is low-rank and we only extract large eigenvalues, i.e. we may get top-T eigenvalues where $T\leq R$. Assume $f(\hat{x})=sgn(g(\hat{x})),f_{\Omega}(\hat{x})=sgn(g_{\Omega}(\hat{x}))$, there exists the error between $g(\hat{x})$ and $g_{\Omega}(\hat{x})$. Therefore, the selection of eigenvalues is crucial (see the appendix C). For this case, we should use the QSVT algorithm and its expansion Duan et al. (2018, 2019); Lin et al. (2019) which can employ threshold to eliminate small eigenvalues. Our algorithm is also suitable for high-rank matrix by running an improved QPCA algorithm Lin et al. (2019) which is based on threshold. Besides, the quantum-inspired algorithm Gilyén et al. (2018); Tang (2018) can handle the low-rank matrices which is helpful to get the target eigenvalues.

The goal of QSLS-SVM algorithm is mainly to obtain the $\alpha_{\Omega}^{{}^{\prime}}$ instead of $\alpha$. This idea may also play an important role in the dual form solution of support vector machines (see appendix D). Obviously, the parameters of our algorithm are one of the solutions of SVM. In summary, we have investigated quantum-enhanced least-square SVM with two quantum algorithms: a simplified quantum algorithm for least-square SVM assisted with continuous variables, and a hybrid quantum-classical procedure that allows sparse solutions for least-square SVM with quantum-enhanced feature maps. The algorithmic complexity of the two algorithms costs $O(\varepsilon^{-1}log(MN))$ and $O(N+R^{3}+R(\varepsilon^{-1}log(MN)))$, respectively, and both give exponential speed-up over the sample size M. The algorithms proposed here may be integrated to tackle classically difficult machine learning tasks with classical intractable quantum feature map.

Appendix A The construction of kernel quantum state: Guass and polynomial kernel functions

In this section we discuss the kernel map in Hilbert space. And in this paper we only describe the polynomial kernel function $K(x,z)=(\langle x|z\rangle+1)^{d}$ and radial basis function $K(x,z)=e^{-|x-z|^{2}/2\omega^{2}}$. For $K(x,z)=(\langle x,z\rangle+1)^{d}$, we take $d=2$ and $x=(x_{1},x_{2}),z=(z_{1},z_{2})$ as a example to analyze this situation. Then we have

[TABLE]

where $\phi(x)=(x_{1}^{2},x_{1}x_{2},x_{1}x_{2},x_{2}^{2},x_{1},x_{2},x_{1},x_{2},1)$ and $\phi(z)$ has similar form. Assume the vector $(x,1)$ can be encoded the $|x\rangle=x_{1}|00\rangle+x_{2}|01\rangle+|10\rangle=\sum_{i=1}^{3}x_{i}|i\rangle$, we have $|\phi(x)\rangle=\sum_{i,j=1}^{3}x_{i}x_{j}|i\rangle|j\rangle$. Finally, we consider a general case. Let the data $x=(x_{1},\cdots,x_{M}),x^{\prime}=(x,1)$ and $|\phi(x)\rangle=\underbrace{|x^{\prime}\rangle\cdots|x^{\prime}\rangle}_{d}=\sum_{k_{1},\cdots,k_{d}=1}^{M+1}x_{k_{1}}\cdots x_{k_{d}}|k_{1}\rangle\cdots|k_{d}\rangle$. Of course, the entries of vector $\phi(x)$ change the order.

[TABLE]

For radial basis function $K(x,z)=e^{-|x-z|^{2}/2\omega^{2}}$, we consider the $K(x,z)=e^{-|x-z|^{2}/2\omega^{2}}=e^{-\sum_{i=1}^{M}(x_{i}-z_{i})^{2}/2\omega^{2}}$. Then we have $K(x,z)=e^{\sum_{i=1}^{M}-x_{i}^{2}/2\omega^{2}}e^{\sum_{i=1}^{M}x_{i}z_{i}/\omega^{2}}e^{\sum_{i=1}^{M}-z_{i}^{2}/2\omega^{2}}$. Inspired by Ref.Rahimi andR̃echt (2008); Talwalkar (2010); Ring andẼskofier (2016), we employ the limitation to solve this separation problem of $h(x,z)=e^{\sum_{i=1}^{M}x_{i}z_{i}/\omega^{2}}=e^{xz^{T}/\omega^{2}}$ which can be denoted as $h(x,z)=\phi(x)\phi(z)^{T}$. The limitation has the following form:

[TABLE]

Thus, there exist a constant $N_{0}$ and accuracy $\delta$ such that $|(1+\frac{s}{k})^{k}-e^{s}|<\delta$ if the integer $k$ satisfies $k>N_{0}$. The function $h(x,z)$ can be approximated as $(1+\frac{xz^{T}/\omega^{2}}{k})^{k}$ with accuracy $\delta$. So we can transform separation problem of $h(x,z)$ to separation problem of $c(x,z)=1+\frac{xz^{T}/\omega^{2}}{k}$. Furthermore, we consider function $c^{\prime}(x,z)=xz^{T}+\omega^{2}k$. Obviously, $c^{\prime}(x,z)$ can be denoted as $c^{\prime}(x,z)=(x,\omega\sqrt{k})(z,\omega\sqrt{k})^{T}$. The vector $x^{\prime}=(x,\omega\sqrt{k})$ can be loaded as a quantum state $|x^{\prime}\rangle$. The final state can be denoted as tensor product of this $k$ quantum states, e.g. $|\phi(x^{\prime})\rangle=\underbrace{|x^{\prime}\rangle\otimes\cdots\otimes|x^{\prime}\rangle}_{k}$. Taking normalization coefficient into account, we have

[TABLE]

where $N_{x^{\prime}}^{-1}=(\frac{xx^{T}+\omega\sqrt{k}}{(\omega\sqrt{k})})^{-k}$ is the normalization factor and satisfies $\mathop{lim}_{k\rightarrow\infty}N_{x^{\prime}}^{-1}=e^{\sum_{i=1}^{M}-x_{i}^{2}/\omega^{2}}$. Eventually, we obtain a quantum state about approximating radial basis function with accuracy $\delta$ and the preparation of Eqs.(A.4) costs time $O(N_{0}log(MN))$. More broadly, the other kernel functions such as exponential and rational quadratic kernel functions can also achieve this expression such as Eqs.(A.4).

However, if we encode $\phi(A)$ as a quantum state $|\phi(A)\rangle=\sum_{i=1}^{M}|\phi(x_{i})||i\rangle|\phi(x_{i})\rangle$, there actually is $|\phi(A)\rangle\propto\phi(A)/e^{\sum_{i=1}^{M}-x_{i}^{2}/2\omega^{2}}$ and this affects our result. In this paragraph, we give instructions. Assume the matrix $\varphi_{A}$ is

[TABLE]

where $x_{i}^{\prime}=(x_{i},\omega\sqrt{2k}),i=1,2,\cdots,M$. From Eqs.(A.5) we can draw a conclusion that the matrix $\varphi_{A}$ is one-sparse and can be decomposed as

[TABLE]

where $\Sigma_{\varphi_{A}}=diag({\sqrt{x_{i}^{\prime}(x_{i}^{\prime})^{T}}})$ and $e_{i}$ represents the i-th column of $I_{M\times M}$. Then we set \Psi=\left(\begin{array}[]{cc}0&\varphi_{A}\\ \varphi_{A}^{T}&0\\ \end{array}\right), $\Psi$ are one-sparse and can be used to construct the unitary transformations $e^{-i\Psi^{1}t}$. The eigenvectors of $\Psi$ can be $(e_{i}^{T},0)^{T},(0,f_{i}^{T})^{T}$. The $(e_{i}^{T},0)^{T}$ can be mapped into the quantum state $|i\rangle$. According to the Ref.[37], we have

[TABLE]

This step can be repeated $k$ times to get the state $\frac{1}{\sqrt{(x_{i}^{\prime}(x_{i}^{\prime})^{T})^{k}}}|i\rangle$, e.g. the target state $(\omega^{2}k)^{-k/2}e^{\sum_{j=1}^{M}-x_{ij}^{2}/2\omega^{2}}|i\rangle$. Thus, the matrix $\phi(A)$ can be prepared as

[TABLE]

As a matter of fact, the selection of $k$ is a important problem and it costs more expensive if $\langle\phi(x_{p_{1}}),\phi(x_{p_{2}})\rangle>1$. However, normalization coefficient can bring better result. Assume $|\phi(x_{p_{1}})\rangle=\frac{1}{\sqrt{N_{p_{1}}}}\sum_{j}\phi(x_{p_{1}})_{j}|j\rangle$, we have $\langle\phi(x_{p_{1}}),\phi(x_{p_{2}})\rangle=\frac{1}{\sqrt{N_{p_{1}}N_{p_{2}}}}\sum_{j}\phi(x_{p_{1}})_{j}\phi(x_{p_{2}})_{j}$ where $N_{p_{i}}=|x_{p_{i}}|,i=1,2$. Because the inequality $\frac{x+y}{2}\geq\sqrt{xy},x,y\geq 0$, we have $\langle\phi(x_{p_{1}}),\phi(x_{p_{2}})\rangle\leq\frac{1}{2}(\frac{1}{N_{p_{1}}}\sum_{j}\phi(x_{p_{1}})_{j}^{2}+\frac{1}{N_{p_{2}}}\sum_{j}\phi(x_{p_{2}})_{j}^{2})\leq 1$. Thus, the Eqs.(A.3) has $e^{s}\leq e$ and it shows we can reduce the size of $N_{0}$ to achieve the accuracy $\delta$. Finally, the Eqs.(A.8) can be used to construct the matrix $K$

[TABLE]

Appendix B Error of HVQ-SVM and Regularzation

In this section, we discuss the influence of error. The hybrid quantum variables method exists the error since we need to define the width $L$ to preserve precision and the error in homodyne detection. In overall procedure of HVQ-SVM algorithm, we call it the fidelity problem and analyze whether the fidelity problem can bring us good effects.

We combine finite squeezing method Zhang et al. (2019b) to introduce the fidelity problem. According to the Ref.Zhang et al. (2019b), the quantum state using finite squeezing method can be represented as

[TABLE]

where $B(Q_{1},Q_{2})\sim e^{-(Q_{1}^{2}+Q_{2}^{2})/2\alpha_{i}^{2}s^{2}}/s\alpha_{i}$, two qumodes $(Q_{1},Q_{2})=(p_{1},p_{2})$ is post-selection, $\alpha_{i}=\eta(\lambda_{i}^{2}+\chi)$,$\lambda_{i}$ is the eigenvalue of the matrix.$\eta$ and $\chi$ are both parameters. $B(Q_{1},Q_{2})$ satisfies $\alpha_{i}^{2}s^{4}\sim\varepsilon_{q}^{-1}$ where $\varepsilon_{q}^{-1}$ is a error. What we want to explore is the effect of $B(Q_{1},Q_{2})$ on $\lambda_{i}$. Taking a classical representation into account, we have

[TABLE]

where $\tau$ is the product of the normalization factor and $A^{+}$ is equal to $|\digamma\rangle$. Changing one form of expression, the formula is

[TABLE]

where

[TABLE]

Function $C(s_{1},\lambda_{i})$ satisfies the inequality:

[TABLE]

Equ.(29) is modified to

[TABLE]

This formula shows that finite squeezing factor regularization is similar to $L_{2}$ regularization, which is a process of reduction along singular value vectors. Our method has same result. Here, we also consider the error of homodyne detection. In HVQ-SVM Algorithm, we can not employ the unitary $U_{p}=I\otimes|0_{p_{1}}\rangle\langle 0_{p_{1}}|\otimes|0_{p_{2}}\rangle\langle 0_{p_{2}}|$ since it can cause this result that the probability of obtaining target state is 0 Arrazola et al. (2018). Assume $|\varepsilon_{q}\rangle=\frac{1}{\pi^{1/4}\sqrt{\varepsilon_{q}}}$, then $U_{p}=I\otimes|\varepsilon_{q}\rangle\langle\varepsilon_{q}|\otimes|\varepsilon_{q}\rangle\langle\varepsilon_{q}|$ and the Eqs.(12) can be improved as

[TABLE]

where

[TABLE]

Then

[TABLE]

where

[TABLE]

Thus, our method has same effect as $L_{2}$ regularization with global phase factor $\hat{F}(\lambda_{1})$ and can cause other effects with different global phase factors $\hat{F}(\lambda_{k})$.

Appendix C Error for $g(\hat{x})$ and $g_{\Omega}(\hat{x})$ in QSLS-SVM algorithm

The expensive cost of obtaining the small eigenvalues leads to the error of ideal output and actual output. In this section, we introduce the error analysis. The error of both functions is

[TABLE]

For the given training data and kernel map $\phi$, the factors of Eqs.(C.1), except for $T$ and $\hat{x}$, is fixed. It shows that we can adjust the threshold $\tau=\sigma_{T}$ to reduce the error. On the other hand, the new data $\hat{x}$ is a uncontrollable factor which has a effect on $E_{\Omega}$ and this is reflected in some data.

Appendix D Apply QSLS-SVM method to soft margin SVM

The dual problem of soft margin SVM model corresponding to Eqs.(5) is

[TABLE]

The Eqs.(17) with the vectors $\alpha_{\Omega}^{T},y_{\Omega}^{T},\mathbf{1}_{\Omega}^{T},K_{\Omega}$ can be simplified to

[TABLE]

The solution of QSLS-SVM algorithm is actually a solution of Eqs.(D.2). And the Eqs.(18) may be helpful for us to solve QP problem. For instance, we can think about ignoring $(\alpha_{\Omega})_{i},i=R+1,\cdots,M$ since these parameters have no effect on the output of SVM.