Classification Problem in a Quantum Framework

TL;DR

This paper develops a quantum version of the Nearest Mean Classifier, introduces a new encoding method for n-dimensional data, and explores how feature rescaling affects its performance through experimental comparisons.

Contribution

It presents a new encoding scheme for quantum classifiers and analyzes the impact of feature rescaling, enhancing the quantum classification framework.

Findings

Quantum NMC performance varies with feature rescaling.

New encoding improves quantum classifier flexibility.

Experimental results compare classical and quantum classifiers.

Abstract

The aim of this paper is to provide a quantum counterpart of the well known minimum-distance classifier named Nearest Mean Classifier (NMC). In particular, we refer to the following previous works: i) in Sergioli et al. 2016, we have introduced a detailed quantum version of the NMC, named Quantum Nearest Mean Classifier (QNMC), for two-dimensional problems and we have proposed a generalization to abitrary dimensions; ii) in Sergioli et al. 2017, the n-dimensional problem was analyzed in detail and a particular encoding for arbitrary n-feature vectors into density operators has been presented. In this paper, we introduce a new promizing encoding of arbitrary n-dimensional patterns into density operators, starting from the two-feature encoding provided in the first work. Further, unlike the NMC, the QNMC shows to be not invariant by rescaling the features of each pattern. This property allows us to introduce a free parameter whose variation provides, in some case, an improvement of the QNMC performance. We show experimental results where: i) the NMC and QNMC performances are compared on different datasets; ii) the effects of the non-invariance under uniform rescaling for the QNMC are investigated.

Tables

Table 1. Table 1: Characteristics of the datasets used in our experiments. The number of instances in each class is shown between brackets.

Data set	Instances	Features ($d$)
Balance	625 (49+288+288)	4
Banana	5300 (2376+2924)	2
Bands	365 (135+230)	19
Breast Cancer (I)	683 (444+239)	10
Breast Cancer (II)	699 (458+241)	9
Ilpd	583 (416+167)	9
Ionosphere	351 (225+126)	34
Liver	578 (413+165)	10
Moon	200 (100+100)	2
Pima	768 (500+268)	8
TicTac	958 (626+332)	9
Gaussian (I)	400 (200+200)	30
Gaussian (II)	1000 (100+900)	8
Gaussian (III)	2050 (50+500+1500)	8

Table 2. Table 2: Comparison between QNMC and NMC performances.

	QNMC
Dataset	E	TPR	TNR	P	K
Balance	0.148 $\pm$ 0.018	0.852 $\pm$ 0.018	0.915 $\pm$ 0.014	0.862 $\pm$ 0.022	0.767 $\pm$ 0.029
Banana	0.316 $\pm$ 0.017	0.684 $\pm$ 0.017	0.660 $\pm$ 0.017	0.684 $\pm$ 0.018	0.350 $\pm$ 0.034
Bands	0.394 $\pm$ 0.053	0.606 $\pm$ 0.053	0.528 $\pm$ 0.071	0.606 $\pm$ 0.058	0.133 $\pm$ 0.112
Breast Cancer (I)	0.386 $\pm$ 0.038	0.614 $\pm$ 0.038	0.444 $\pm$ 0.045	0.583 $\pm$ 0.044	0.062 $\pm$ 0.069
Breast Cancer (II)	0.040 $\pm$ 0.015	0.946 $\pm$ 0.023	0.986 $\pm$ 0.016	0.993 $\pm$ 0.009	0.912 $\pm$ 0.033
Ilpd	0.351 $\pm$ 0.037	0.649 $\pm$ 0.037	0.705 $\pm$ 0.056	0.734 $\pm$ 0.041	0.292 $\pm$ 0.073
Ionosphere	0.165 $\pm$ 0.049	0.835 $\pm$ 0.049	0.764 $\pm$ 0.059	0.842 $\pm$ 0.051	0.624 $\pm$ 0.105
Liver	0.342 $\pm$ 0.037	0.607 $\pm$ 0.057	0.783 $\pm$ 0.059	0.870 $\pm$ 0.039	0.318 $\pm$ 0.061
Moon	0.156 $\pm$ 0.042	0.857 $\pm$ 0.063	0.831 $\pm$ 0.066	0.841 $\pm$ 0.066	0.683 $\pm$ 0.085
Pima	0.304 $\pm$ 0.030	0.696 $\pm$ 0.030	0.690 $\pm$ 0.044	0.720 $\pm$ 0.030	0.365 $\pm$ 0.066
Tic Tac	0.410 $\pm$ 0.032	0.590 $\pm$ 0.032	0.597 $\pm$ 0.039	0.629 $\pm$ 0.036	0.172 $\pm$ 0.061
Gaussian (I)	0.274 $\pm$ 0.051	0.726 $\pm$ 0.051	0.728 $\pm$ 0.049	0.745 $\pm$ 0.048	0.452 $\pm$ 0.099
Gaussian (II)	0.210 $\pm$ 0.025	0.790 $\pm$ 0.025	0.744 $\pm$ 0.061	0.900 $\pm$ 0.019	0.308 $\pm$ 0.058
Gaussian (III)	0.401 $\pm$ 0.036	0.599 $\pm$ 0.036	0.558 $\pm$ 0.026	0.654 $\pm$ 0.041	0152 $\pm$ 0.043
	NMC
Dataset	E	TPR	TNR	P	K
Balance	0.267 $\pm$ 0.038	0.733 $\pm$ 0.038	0.969 $\pm$ 0.014	0.925 $\pm$ 0.025	0.686 $\pm$ 0.034
Banana	0.453 $\pm$ 0.019	0.548 $\pm$ 0.019	0.552 $\pm$ 0.020	0.556 $\pm$ 0.020	0.098 $\pm$ 0.038
Bands	0.435 $\pm$ 0.048	0.565 $\pm$ 0.048	0.582 $\pm$ 0.055	0.605 $\pm$ 0.054	0.135 $\pm$ 0.092
Breast Cancer (I)	0.442 $\pm$ 0.037	0.558 $\pm$ 0.037	0.464 $\pm$ 0.046	0.551 $\pm$ 0.039	0.022 $\pm$ 0.076
Breast Cancer (II)	0.042 $\pm$ 0.015	0.973 $\pm$ 0.015	0.931 $\pm$ 0.032	0.963 $\pm$ 0.017	0.908 $\pm$ 0.033
Ilpd	0.470 $\pm$ 0.037	0.530 $\pm$ 0.037	0.757 $\pm$ 0.041	0.761 $\pm$ 0.037	0.193 $\pm$ 0.051
Ionosphere	0.323 $\pm$ 0.051	0.677 $\pm$ 0.051	0.676 $\pm$ 0.051	0.680 $\pm$ 0.051	0.351 $\pm$ 0.102
Liver	0.472 $\pm$ 0.048	0.388 $\pm$ 0.057	0.891 $\pm$ 0.055	0.905 $\pm$ 0.045	0.193 $\pm$ 0.060
Moon	0.234 $\pm$ 0.065	0.772 $\pm$ 0.089	0.762 $\pm$ 0.085	0.771 $\pm$ 0.091	0.528 $\pm$ 0.130
Pima	0.375 $\pm$ 0.033	0.625 $\pm$ 0.033	0.546 $\pm$ 0.045	0.622 $\pm$ 0.037	0.173 $\pm$ 0.075
Tic Tac	0.439 $\pm$ 0.031	0.561 $\pm$ 0.031	0.571 $\pm$ 0.042	0.606 $\pm$ 0.036	0.119 $\pm$ 0.063
Gaussian (I)	0.322 $\pm$ 0.042	0.679 $\pm$ 0.042	0.680 $\pm$ 0.043	0.685 $\pm$ 0.042	0.355 $\pm$ 0.085
Gaussian (II)	0.320 $\pm$ 0.032	0.680 $\pm$ 0.032	0.588 $\pm$ 0.102	0.860 $\pm$ 0.032	0.129 $\pm$ 0.055
Gaussian (III)	0.530 $\pm$ 0.029	0.470 $\pm$ 0.029	0.625 $\pm$ 0.030	0.620 $\pm$ 0.036	0.066 $\pm$ 0.044

Equations

$${\vec{\mu }}_l=\frac{1}{N_l}\sum _{n=1}^{N_l}{\vec{x}}_n,l=1,2,\dots ,L;$$

$$\underset{l=1,\dots L}{\mathrm{argmin}}{d}_E(\vec{x},{\vec{\mu }}_l),$$

$$SP^{-1}:[x^{(1)},x^{(2)}]\mapsto\Big{[}\frac{2x^{(1)}}{||x||^{2}+1},\frac{2x^{(2)}}{||x||^{2}+1},\frac{||x||^{2}-1}{||x||^{2}+1}\Big{]},$$

$$\frac{1}{2}\left(\begin{array}{cc}1+r_3& r_1-i{r}_2\\ r_1+i{r}_2& 1-r_3\end{array}\right)=\frac{1}{||x|{|}^2+1}\left(\begin{array}{cc}||x|{|}^2& x^{(1)}-i{x}^{(2)}\\ x^{(1)}+i{x}^{(2)}& 1\end{array}\right).$$

$${\vec{x}}^{\prime }=S{P}^{-1}(\vec{x})=\frac{1}{||\vec{x}|{|}^2+1}\left[2x^{(1)},\dots ,2x^{(d)},||\vec{x}|{|}^2-1\right]$$

$$\vec{x}=[x^{(1)},\dots ,x^{(d)}]\mapsto {\vec{x}}^{\prime }=[x^{(1)},\dots ,x^{(d)},||\vec{x}||].$$

$$\vec{x}^{\prime}\ \mapsto\ \vec{x}^{\prime\prime}=\Big{[}\frac{x^{(1)}}{||\vec{x}||},\ldots,\frac{x^{(d)}}{||\vec{x}||},||\vec{x}||\Big{]}.$$

$$\vec{x}^{\prime\prime}\ \mapsto\ \vec{x}^{\prime\prime\prime}=\frac{\vec{x}^{\prime\prime}}{||\vec{x}^{\prime\prime}||}=\Big{[}\frac{x^{(1)}}{||\vec{x}||\sqrt{||\vec{x}||^{2}+1}},\ldots,\frac{x^{(d)}}{||\vec{x}||\sqrt{||\vec{x}||^{2}+1}},\frac{||\vec{x}||}{\sqrt{||\vec{x}||^{2}+1}}\Big{]}.$$

$${\rho }_{\vec{x}}\doteq {\vec{x}}^{\prime \prime \prime }\cdot ({\vec{x}}^{\prime \prime \prime }{)}^{†}$$

$${\mathcal{S}}^q={\mathcal{S}}_{\text{tr}}^q\cup {\mathcal{S}}_{\text{ts}}^q,{\mathcal{S}}_{\text{tr}}^q=\{({\rho }_{{\vec{x}}_n},y_n){\}}_{n=1}^N,{\mathcal{S}}_{\text{ts}}^q=\{({\rho }_{{\vec{x}}_n},y_n){\}}_{n=N+1}^{N^{\prime }},$$

$${\rho }_l=\frac{1}{N_l}\sum _{n=1}^{N_l}{\rho }_{{\vec{x}}_n},l=1,\dots ,L$$

$${\rho }_l\ne {\rho }_{{\vec{\mu }}_l},\forall l=1,\dots ,L.$$

$$d_T(\rho ,{\rho }^{\prime })=\frac{1}{2}\mathrm{Tr}|\rho -{\rho }^{\prime }|,$$

$$\underset{l=1,\dots ,L}{\mathrm{argmin}}{d}_T({\rho }_{\vec{x}},{\rho }_l),$$

$$\underset{l=1,\dots ,L}{\mathrm{argmin}}{d}_E(t{\vec{x}}_n,t{\vec{\mu }}_l)=t\underset{l=1,\dots ,L}{\mathrm{argmin}}{d}_E({\vec{x}}_n,{\vec{\mu }}_l),\forall n=N+1,\dots ,N^{\prime }.$$

Full text

1 Introduction

The aim of this paper is to propose a classification algorithm inspired by quantum theory. In particular, the classification process we propose consists in the following steps: $i)$ we encode a dataset of reals objects (represented by real vectors) into quantum objects (represented by density operators); $ii)$ we perform a particular classification process of quantum objects by involving a standard notion of distance between quantum states (the trace distance); $iii)$ we naturally decode the results of the classification of quantum objects for the initial real dataset.

The result is that we can perform (by using a classical computer) a classification of real objects by involving the formalism of quantum mechanics. Interstingly enough, the error of this quantum-inspired encoding turns out to be smaller with respect to the well known nearest mean classifier for many different datasets.

However, some attempts to represent a classification process by appealing to the quantum theory was already realized aimeur2006 ; 12 ; 13 ; Rebentrost ; sergioli2017 ; sergioli2016 . In particular, the problem to find a more convenient encoding from classical to quantum object is nowaday an open and interesting problem 10 ; 8 .

In this work we propose a new encoding that leads to three relevant advantages: i) it allows to simply encode an arbitrary $n$-feature pattern into a quantum state; ii) the classification process performance turns out to be better than the standard NMC for several and different kinds of datasets; ii) for some dataset, this new encoding exhibits a further advantage that can be gained by exploiting the non-invariance under rescaling of the quantum-inspired classifier.

The paper is organized as follows: in Section 2 we briefly describe the classification process and, in particular, the formal structure of the NMC. Section 3 is devoted to the definition of a new encoding of real patterns into quantum states. In Section 4 we introduce the quantum version of the NMC based on the new encoding previously described. In Section 5 we compare the NMC and the QNMC on different datasets showing that, in general, the QNMC exhibits better performances (in terms of accuracy and other significant statistical quantities) with respect to the NMC. Further, starting from the fact that, differently from the NMC, the QNMC is not invariant under rescaling, we also show that, for some dataset, it is possible to provide a benefit from this non-invariance property. Some conclusions and possible further developments are proposed at the end of the paper.

2 On the classification process

Here, we address the classification problem, which is an instance of supervised learning, i.e. learning from a training set of correctly labeled objects. More precisely, each object can be characterized by its features; hence, a $d$-feature object can be naturally represented by a $d$-dimensional real vector, i.e. $\vec{x}=[x^{(1)},\ldots,x^{(d)}]\in\mathcal{X}$, where $\mathcal{X}\subseteq\mathbb{R}^{d}$ is generally a subset of the $d$-dimensional real space and represents the feature space. Hence, any arbitrary object is represented by a vector $\vec{x}$ associated to a given class of objects (but, in principle, we do not know which one). Let $\mathcal{Y}=\{1,\ldots,L\}$ be the class label set. A pattern is represented by a pair $(\vec{x},y)$, where $\vec{x}$ is the feature vector representing an object and $y\in\mathcal{Y}$ is the label of the class which $\vec{x}$ is associated to. The aim of the classification process is to identify which class $\vec{x}$ belongs to, by learning about the set of objects whose class is known. Then, the so called training set is given by $\mathcal{S}_{\text{tr}}=\{({\bf x}_{n},y_{n})\}_{n=1}^{N}$, where ${\bf x}_{n}\in\mathcal{X}$, $y_{n}\in\mathcal{Y}$ ($\forall n=1,\ldots,N$) and $N$ is the number of patterns belonging to $\mathcal{S}_{\text{tr}}$. Finally, let $N_{l}$ be the cardinality of the training set associated to the $l$-th class (for $l=1,2,\ldots,L$) such that $\sum_{l=1}^{L}N_{l}=N$.

We now introduce the well known Nearest Mean Classifier (NMC) DuHa , which is a particular kind of minimum-distance classifier widely used in pattern recognition. The strategy consists in computing the distances between a pattern $\bf x$ (to classify) and other patterns chosen as prototypes of each class (called centroids). Finally, $\bf x$ is labeled as belonging to the class whose distance is minimum. So, we can resume the NMC algorithm as follows:

The computation of the centroid (i.e. the sample mean Johnson ) associated to each class is given by:

[TABLE] 2. 2.

The classification of the pattern $\vec{x}$ is provided by:

[TABLE]

where $d_{E}(\vec{x},\vec{\mu}_{l})=\|\vec{x}-\vec{\mu}_{l}\|$ is the Euclidean distance between the pattern $\vec{x}$ and the centroid $\vec{\mu}_{l}$.

Briefly speaking, if the pattern $\vec{x}$ is closest to the centroid $\vec{\mu}_{l}$, then $\vec{x}$ is labeled by $l$, i.e. $y=l$.

As a remark, it is worth noting that, depending on the particular distribution of the patterns of the dataset, it is possible that a pattern belonging to a given class is closest to the centroid of another class. In this case, if the algorithm would be applied to this pattern, it would fail. Hence, for an arbitrary pattern $\vec{x}$ whose class is a priori unknown, the output of above classification process has the following four possibilities F : i) True Positive (TP): pattern belonging to the $l$-th class and correctly classified as $l$; ii) True Negative (TN): pattern belonging to a class different than $l$, and correctly classified as not $l$; iii) False Positive (FP): pattern belonging to a class different than $l$, and uncorrectly classified as $l$; iv) False Negative (FN): pattern belonging to the $l$-th class, and uncorrectly classified as not $l$.

In order to evaluate the performance of a certain classification algorithm, the standard procedure consists in dividing the original labeled dataset $\mathcal{S}$ of $N^{\prime}$ patterns, into a training set $\mathcal{S}_{\text{tr}}$ of $N$ patterns and a set $\mathcal{S}_{\text{ts}}$ of $(N^{\prime}-N)$ patterns (i.e. $\mathcal{S}=\mathcal{S}_{\text{tr}}\cup\mathcal{S}_{\text{ts}}$). This set $\mathcal{S}_{\text{ts}}$ of patterns is called test set DuHa and it is defined as $\mathcal{S}_{\text{ts}}=\{({\bf x}_{n},y_{n})\}_{n=N+1}^{N^{\prime}}$, where $N^{\prime}_{l}$ (such that $\sum_{l=1}^{L}N^{\prime}_{l}=N^{\prime}-N$) is the cardinality of the test set associated to the $l$-th class.

Then, by applying the NMC to the test set, it is possible to evaluate the classification algorithm performance by considering the following statistical measures associated to each class $l$ depending on the quantities listed above:

• •

True Positive Rate (TPR): $\text{TPR}=\frac{\text{TP}}{\text{TP}+\text{FN}}$;

• •

True Negative Rate (TNR): $\text{TNR}=\frac{\text{TN}}{\text{TN}+\text{FP}}$;

• •

False Positive Rate (FPR): $\text{FPR}=\frac{\text{FP}}{\text{FP}+\text{TN}}=1-\text{TPN}$;

• •

False Negative Rate (FNR): $\text{FNR}=\frac{\text{FN}}{\text{FN}+\text{TP}}=1-\text{TPR}$.

Further, other standard statistical indeces F used to establish the reliability of a classification algorithm are:

• •

Classification error (E): $\text{E}=1-\frac{\text{TP}}{\text{N}^{\prime}-\text{N}}$;

• •

Precision (P): $\text{P}=\frac{\text{TP}}{\text{TP}+\text{FP}}$;

• •

k’s Cohen (K): $\text{K}=\frac{\text{Pr(a)}-\text{Pr(e)}}{1-\text{Pr(e)}}$, where

$\text{Pr(a)}=\frac{\text{TP}+\text{TN}}{\text{N}^{\prime}-\text{N}}$, $\text{Pr(e)}=\frac{(\text{TP}+\text{FP})(\text{TP}+\text{FN})+(\text{FP}+\text{TN})(\text{TN}+\text{FN})}{(\text{N}^{\prime}-\text{N})^{2}}$.

In particular, the classification error represents the percentage of misclassified patterns, the precision is a measure of the statistical variability of the considered model and the k’s Cohen represents the degree of agreement among items that can assume values ranging from $-1$ to $+1$ (K $=+1$ corresponds to a perfect classification procedure while K $=-1$ corresponds to a completely wrong classification). Let us note that these statistical parameters have to be a part considered for each class. Then, the final value of each statistical parameter related to the classification algorithm is the weighted sum of the statistical parameters of each class.

3 Correspondence between pattern and density operator

In order to introduce a quantum version of the NMC, the first step is to find an appropriate quantum encoding for a real pattern.

Generally, given a $d$-dimensional feature vector, there exist different ways to encode it into a density operator 8 . In sergioli2016 the following encoding was introduced. Let us consider the inverse of the steregraphic projection Cox given by:

[TABLE]

where $||x||^{2}=[x^{(1)}]^{2}+[x^{(2)}]^{2}$. Then, by imposing that $r_{1}=\frac{2x^{(1)}}{||x||^{2}+1},\ r_{2}=\frac{2x^{(2)}}{||x||^{2}+1},\ r_{3}=\frac{||x||^{2}-1}{||x||^{2}+1}$, if we consider $r_{1},r_{2},r_{3}$ as Pauli components111We consider the representation of an arbitrary density operator as linear combination of Pauli matrices. of a density operator $\rho_{\bf x}\in\mathbb{C}^{2}$, then the density operator associated to the pattern $\vec{x}=[x^{(1)},x^{(2)}]$ can be written as:

[TABLE]

The advantage in using this encoding consists in the fact that it provides an easy visualization of an arbitrary two-feature vector on the Bloch sphere sergioli2016 . However, the main problem regards the generalization for $d$-feature vectors. Although in sergioli2016 a generalization to the $d$-feature case was introduced, it exhibits some difficulties to be implemented for general cases.

An alternative encoding of a $d$-feature pattern $\vec{x}=[x^{(1)},\ldots,x^{(d)}]$ into a density operator was proposed in sergioli2017 . It is obtained i) by mapping $\vec{x}\in\mathbb{R}^{d}$ into a ($d+1$)-dimensional vector $\vec{x}^{\prime}\in\mathbb{R}^{d+1}$ achieved by the generalized Eq. (3), i.e.

[TABLE]

where $||\vec{x}||^{2}=\sum_{i=1}^{d}[x^{(i)}]^{2}$, and then ii) by considering the projector $\rho_{\vec{x}}=\vec{x}^{\prime}\cdot(\vec{x}^{\prime})^{T}$.

In this work we propose a different version of the QNMC based on a new encoding again and we show that this exhibits interesting improvements mostly by exploiting the non invariance under rescaling of the features.

Accordingly with 12 ; Rebentrost ; 8 , when a real vector is encoded into a quantum state, in order to avoid a loss of information it is important that the quantum state keeps some information about the norm of the original real vector. In light of this fact, we introduce the following alternative encoding.

Let $\vec{x}=[x^{(1)},\ldots,x^{(d)}]\in\mathbb{R}^{d}$ be an arbitrary $d$-feature vector.

We maps the vector $\vec{x}\in\mathbb{R}^{d}$ into a vector $\vec{x}^{\prime}\in\mathbb{R}^{d+1}$, whose first $d$ features are the components of the vector $\vec{x}$ and the $(d+1)$-th feature is the norm of $\vec{x}$. Formally:

[TABLE] 2. 2.

Finally, we obtain the vector $\vec{x}^{\prime}$ by dividing the first $d$ components of the vector $\vec{x}^{\prime}$ for $||\vec{x}||$:

[TABLE] 3. 3.

We consider the norm of the vector $\vec{x}^{\prime\prime}$, i.e. $||\vec{x}^{\prime\prime}||=\sqrt{||\vec{x}||^{2}+1}$ and we map the vector $\vec{x}^{\prime\prime}$ into the normalized vector $\vec{x}^{\prime\prime\prime}$ as follows:

[TABLE]

Now, we provide the following definition.

Definition 1 (Density pattern)

Let $\vec{x}=[x^{(1)},\ldots,x^{(d)}]$ be an arbitrary $d$-features pattern. Then, the matrix representation of the density pattern $\rho_{\vec{x}}$ corresponding to the pattern $\vec{x}$ is defined as:

[TABLE]

where $\vec{x}^{\prime\prime\prime}$ is given by Eq (8).

Hence, this encoding maps real $d$-dimensional vectors ${\vec{x}}$ into $(d+1)$-dimensional pure states $\rho_{\vec{x}}$. In this way, we obtain an encoding that takes into account the information about the initial real vector norm and, in the meantime, allows to easily encode also arbitrary $d$-dimensional real patterns.

4 Quantum classification

In this section we introduce a quantum-inspired version of the NMC, named Quantum Nearest Mean Classifier (QNMC). It can be seen as a particular kind of minimum-distance classifier between quantum objects (i.e. density patterns). The use of this new formalism could lead not only to achieve the well known advantages related to the quantum computation with respect to the classical one (mostly related to the speed up of the computation process), but also to make a full comparison between NMC and QNMC performance by using a classical computer only.

In order to provide a quantum counterpart of the NMC, we need: i) an encoding from real patterns to quantum objects (already defined in the previous section); ii) a quantum counterpart of the classical centroid (i.e. a sort of class prototype), that will be named quantum centroid; iii) a suitable definition of quantum distance between density patterns, that plays the same role as the Euclidean distance for the NMC. In this quantum framework, the quantum version $\mathcal{S}^{q}$ of the dataset $\mathcal{S}$ is given by:

[TABLE]

where $\rho_{\vec{x}_{n}}$ is the density pattern associated to the pattern $\vec{x}_{n}$ and $y_{n}$ is its original label. Then, $\mathcal{S}^{q}_{\text{tr}}$ and $\mathcal{S}^{q}_{\text{ts}}$ represent the quantum versions of training and test set respectively, i.e. the sets of all the density patterns obtained by encoding all the elements of $\mathcal{S}_{\text{tr}}$ and $\mathcal{S}_{\text{ts}}$. Now, we naturally introduce the quantum version of the classical centroid $\vec{\mu}_{l}$, given in Eq. (1), as follows.

Definition 2 (Quantum centroid)

Let $\mathcal{S}^{q}$ be a labeled dataset of $N^{\prime}$ density patterns such that $\mathcal{S}^{q}_{\text{tr}}\subseteq\mathcal{S}^{q}$ is a training set composed of $N$ density patterns. Further, let $\mathcal{Y}=\{1,2,\ldots,L\}$ be the set of class labels. The quantum centroid of the $l$-th class is given by:

[TABLE]

where $N_{l}$ is the number of density patterns of the $l$-th class belonging to $\mathcal{S}^{q}_{\text{tr}}$, such that $\sum_{l=1}^{L}N_{l}=N$.

Notice that the quantum centroids are generally mixed states and they are not obtained by encoding the classical centroids $\vec{\mu}_{l}$, i.e.

[TABLE]

Accordingly, the definition of the quantum centroid leads to a new object that is no longer a pure state and has not any classical counterpart. This is the main reason that establishes, even in a foundamental level, the difference between NMC and QNMC. In particular, it is easy to verify sergioli2016 that, unlike the classical case, the expression of the quantum centroid is sensitive to the dataset dispersion.

In order to consider a suitable definition of distance between density patterns, we recall the well known definition of trace distance between quantum states (see, e.g. nielsenbook ).

Definition 3 (Trace distance)

Let $\rho$ and $\rho^{\prime}$ be two quantum density operators belonging to the same dimensional Hilbert space. The trace distance between them is given by:

[TABLE]

where $|A|=\sqrt{A^{\dagger}A}$.

Notice that the trace distance is a true metric for the density operators, that is, it satisfies: i) $d_{T}(\rho,\rho^{\prime})\geq 0$ with equality iff $\rho=\rho^{\prime}$ (positivity), ii) $d_{T}(\rho,\rho^{\prime})=d_{T}(\rho^{\prime},\rho)$ (symmetry) and iii) $d_{T}(\rho,\rho^{\prime})+d_{T}(\rho^{\prime},\rho^{\prime\prime})\geq d_{T}(\rho,\rho^{\prime\prime})$ (triangle inequality).

We have introduced all the ingredients we need to describe the QNMC process, that, similarly to the classical case, consists in the following steps:

• •

to construct quantum training and test sets $\mathcal{S}^{q}_{\text{tr}}$, $\mathcal{S}^{q}_{\text{ts}}$ by applying the encoding introduced in Definition 1 to each pattern of the classical training and test sets $\mathcal{S}_{\text{tr}}$, $\mathcal{S}_{\text{ts}}$;

• •

to calculate the quantum centroids $\rho_{l}$ ($\forall l=1,\ldots L$), by using the quantum training set $\mathcal{S}^{q}_{\text{tr}}$, according to Definition 2;

• •

to classify an arbitrary density pattern $\rho_{\vec{x}}\in S^{q}_{\text{ts}}$ accordingly with the following minimization problem:

[TABLE]

where $d_{T}$ is the Trace distance introduced in Definition 3.

5 Experimental results

This section is devoted to show a comparison between the NMC and the QNMC performances in terms of the statistical parameters introduced in Section 2. We use both classifiers to analyze fourteen datasets. In particular, two different kinds of datasets have been studied: five of them (Gaussian (I), Gaussian (II), Gaussian (III), Moon, Banana) are artificial datasets (in particular, the first three datasets follow Gaussian distributions), while the others (Balance, Bands, Breast Cancer (I), Breast Cancer (II), Ilpd, Ionosphere, Liver, Pima, Tic Tac) are real-world datasets, extracted from the UCI repository222http://archive.ics.uci.edu/ml and following unknown distributions. We stress that, in real situations, we usually deal with datasets following unknown distributions, then the most interesting case is the second one. However, the use of artificial datasets following known distribution, and in particular Gaussian distributions with specific parameters, can help to catch precious information that will be discussed in the next section.

5.1 Comparison between QNMC and NMC

In Table 1 we summarize the characteristics of the datasets involved in our experiments. In particular, for each dataset we list the total number of patterns, the number of patterns belonging to each class and the number of features. Let us note that, although we mostly confine our investigation to two-class datasets, our model can be easily extended -without any loss of generality- to multiclass problems (as we show for the three-class datasets Balance and Gaussian (III)).

In order to make our results statistically significant, we apply the standard procedure of splitting each dataset in training and test sets composed of the $\%80$ and $\%20$ of the total patterns respectively, and we carry out ten experiments for each dataset, where the splitting is every time randomly taken.

In Table 2, we report QNMC and NMC performance for each dataset, evaluated in terms of mean value and standard deviation (computed on ten runs) of the statistical indexes, discussed in the previous section. For the sake of semplicity, we omit the values of FPR and FNR because they can be easily obtained by TPR and TNR values (i.e. FPR = 1 - TNR, FNR = 1 - TPR).

We observe, by comparing QNMC and NMC performances (see Table 2), that the first provides a significant improvement with respect to the standard NMC in terms of all the statistical parameters we have considered. Further, the new encoding, for two-feature datasets, provides better performance than the one considered in sergioli2016 (where the QNMC error with related standard deviation was $0.174\pm 0.047$ for Moon and $0.419\pm 0.015$ for Banana) and it generally exhibits quite similar performance with respect to the one in sergioli2017 for multi-dimension datasets, except in the case of Breast Cancer (II) and Gaussian (I) datasets, for which the new encoding provides a classification improvement of about $3\%$ and $5\%$, respectively.

The artificial Gaussian datasets may deserve a brief comment. Let us discuss the way in which the three Gaussian datasets have been created. The first one, called Gaussian (I) Skurichina02 is a perfectly balanced dataset (i.e. both classes have the same number of patterns), patterns have the same dispersion in both classes, and only some features are correlated Wass . The second one, called Gaussian (II), is an unbalanced dataset (i.e. classes have a very different number of patterns), patterns have not the same dispersion in both classes and features are not correlated. Finally, the third one, called Gaussian (III), is composed of three classes and it is an unbalanced dataset with different pattern dispersion in all the classes, where all the features are correlated.

For these Gaussian datasets, the NMC is not the best classifier DuHa because of the particular characteristics of the class dispersion. Indeed, the NMC it does not take into account of the data dispersion. Conversely, by looking at Table 2, the improvements of the QNMC seems to exhibit some kind of sensitivity of the classifier with respect to the data dispersion. A detailed description of this problem will be addressed in a future work.

As a remark, it is important to remind that, even if it is possible to establish which is a good or bad classifier for a given dataset by the evaluation of some a priori data characteristics, generally it is no possible to establish an absolute superiority of a given classifier for any dataset, according to the well known No Free Lunch Theorem DuHa . Anyway, the QNMC seems to be particularly convenient when the data distribution is difficult to treat with the standard NMC.

5.2 Non-invariance under rescaling

The final experimental results that we present in this paper regard a significant difference between NMC and QNMC. Let us suppose that all the features of the patterns $\vec{x}_{n}$ ($\forall n=1,\ldots,N^{\prime}$) belonging to the original dataset $\mathcal{S}$ are multiplied by the same parameter $t\in\mathbb{R}$, i.e. $\vec{x}_{n}\mapsto t\vec{x}_{n}$. Then, the whole dataset is subjected to an increasing dispersion (for $|t|>1$) or a decreasing dispersion (for $|t|<1$) and the classical centroids change according to $\vec{\mu}_{l}\mapsto t\vec{\mu}_{l}$ ($\forall l=1,\ldots,L$). Consequently, the classification problem for each pattern of the rescaled test set can be written as

[TABLE]

For any value of the parameter $t$ it can be proved sergioli2017 that, while the NMC is invariant under rescaling, for the QNMC this invariance does no longer hold. Interestingly enough, it is possible to evaluate this interesting property of the QNMC as an advantage for the classification process. In other words, by a suitable choise of the rescaling factor is possible, in principle, to get a decreasing of the classification error. At this purpose, we have studied the variation of the QNMC performance (in particular of the classification error) in terms of the free parameter $t$ and in Fig. 1 the results for the datasets Ionosphere and Bands are shown. In the figure, each point represents the mean value (with related standard deviation represented by the vertical bar) over ten runs of the experiments. We can observe that, for the considered datasets, the QNMC performance for the most of $t$ values is better than the NMC, but for some particular value of $t$ the error gets a further significant reduction (with respect the unrescaled case).

Let us note that the range of the rescaling parameter $t$ for which the QNMC performance improves, is generally not unique and depends on the dataset. For instance, in Fig. 1, we observe that the classification error provided by the QNMC decreases for $t$ ranging from $0.1$ to $1.9$, in the Ionosphere case, and from $0.001$ to $0.019$ in the Bands case. Then, we do not generally get an improvement in the classification process for any $t$ ranges. On the contrary, there exist some intervals of the parameter $t$ where the QNMC classification performance is worse than the case without rescaling. Then, each dataset has specific and unique characteristics (in accord to the No Free Lunch Theorem) and the incidence of the non invariance under rescaling in the decresing of the error is, in general, to determinate by empirical evidences.

6 Conclusions and future work

In this work a quantum counterpart of the well known Nearest Mean Classifier has been proposed. We have introduced a quantum minimum distance classifier, called Quantum Nearest Mean Classifier, obtained by defining a suitable encoding of real patterns, i.e. density patterns, and by recovering the trace distance between density operators.

We propose a new encoding of real pattern into quantum object that was suggested by recent debates on quantum machine learning according to which, in order to avoid a loss of information caused by encoding a real vector into a quantum state, we need to normalize the real vector mantaining some information about its norm. Secondly, we defined the quantum centroid, i.e. the pattern chosen as the prototype of each class, which is not invariant under uniform rescaling of the original dataset (unlike the NMC) and seems to exhibit a kind of sensitivity to the data dispersion.

The experiments are organized as follows: both classifiers have been compared in terms of significant statistical indeces. In particular, we considered fourteen different datasets having different nature (real-world and artificial). Further, the no-invariance under rescaling of the QNMC suggested to study the variation of the classification error in terms of a free parameter $t$, whose variation produces a modification of the data dispersion and, consequently, of the classifier performance. In particular we have shown as, in the most of the cases, the QNMC exhibits a significant decreasing of the classification error (and of the other statistical parameters) with respect of the NMC and, for some case, the non invariance under rescaling can provide a significant positive incidence in the classification process.

Let us remark that, even if there is not an absolute superiority of QNMC with respect to the NMC, the method we introduced allows to get some relevant improvements of the classification when we have an a priori knowledge about the distribution of the dataset we have to deal with.

In light of such considerations, further developments of the present work will be focused on: i) finding out the encoding (from real vectors to density operators) that guarantees the optimal improvement (at least for a finite class of datasets) in terms of the classification process accuracy; ii) obtain a general method to find the suitable rescaling parameter range to apply to a given dataset in order to get a further improvement of the accuracy; iii) understanding for which kind of distribution the QNMC performs better than the NMC. At this purpose, it will be useful to compare the optimal QNMC also with other standard classical classifiers.

Acknowledgements.

This work is supported by the Sardinia Region Project “Time-logical evolution of correlated microscopic system”, LR 7/8/2007 (2015). RAS CRP-55. GMB acknowledges support from CONICET and UNLP (Argentina).