On Reject and Refine Options in Multicategory Classification

TL;DR

This paper introduces margin-based multicategory classification methods with reject and refine options, improving error identification and providing more informative predictions, supported by theoretical analysis and empirical validation.

Contribution

It proposes novel reject and refine options for multicategory classification, with theoretical guarantees and efficient algorithms, extending binary reject options to more complex settings.

Findings

Effective identification of error-prone observations.

Refine option improves classification by ruling out unlikely classes.

Methods outperform standard multicategory classifiers in simulations and real data.

Abstract

In many real applications of statistical learning, a decision made from misclassification can be too costly to afford; in this case, a reject option, which defers the decision until further investigation is conducted, is often preferred. In recent years, there has been much development for binary classification with a reject option. Yet, little progress has been made for the multicategory case. In this article, we propose margin-based multicategory classification methods with a reject option. In addition, and more importantly, we introduce a new and unique refine option for the multicategory problem, where the class of an observation is predicted to be from a set of class labels, whose cardinality is not necessarily one. The main advantage of both options lies in their capacity of identifying error-prone observations. Moreover, the refine option can provide more constructive information for classification by effectively ruling out implausible classes. Efficient implementations have been developed for the proposed methods. On the theoretical side, we offer a novel statistical learning theory and show a fast convergence rate of the excess $\ell$-risk of our methods with emphasis on diverging dimensionality and number of classes. The results can be further improved under a low noise assumption. A set of comprehensive simulation and real data studies has shown the usefulness of the new learning tools compared to regular multicategory classifiers. Detailed proofs of theorems and extended numerical results are included in the supplemental materials available online.

Tables

Table 1. Table 1: Simulation results for Examples 1, 2 and 3, with the Soft loss, the SVM loss and the DWD loss respectively. The average proportion and misclassification or mis-refinement rate on each observation set p 1 subscript 𝑝 1 p_{1} , p 2 subscript 𝑝 2 p_{2} or p 3 subscript 𝑝 3 p_{3} for each of the three classifiers (regular, with rejection only, and with both reject and refine options) are reported over 100 replications. A probability estimation method for reject option is also compared. The overall empirical 0 0 - d 𝑑 d - 1 1 1 loss is reported for each classifier and each example. Proportions for selected prediction sets are reported as well. The results show improved overall loss by our methods and successful identification of a subset of most confusing classes.

Example 1, Soft with $a=a_2$			Regular	Reject	R&R	Probability Method
	Proportion		Error			Proportion	Error
p1	49.43		28.80	27.58	27.58	52.26	30.17
p2	28.97	size 2: 24.62	45.89	45.35	1.581
		size 3: 4.349
p3	21.60		69.61	-	-	47.74	-
Overall	100.0		41.92	39.32	27.47	100.0	45.64
Example 2, SVM with $a=a_1$			Regular	Reject	R&R	Probability Method
	Proportion		Error			Proportion	Error
p1	52.53		28.15	27.81	27.81	42.40	25.85
p2	size2: 12.90		48.57	51.00	11.26
	$\lfloor \{1,2\}$: 73.1%
p3	34.57		53.01	-	-	57.60	-
Overall	100.0		39.57	38.91	33.33	100.0	40.66
Example 3, DWD with $a=a_2$			Regular	Reject	R&R	Probability Method
	Proportion		Error			Proportion	Error
p1	45.58		25.68	26.11	26.11	36.97	24.72
p2	23.26	size 2: 19.71	36.45	35.71	1.771
		$\lfloor \{1,2\}$: 40.3%
		$\lfloor \{3,4\}$: 42.9%
		size 3: 3.549
p3	31.16		56.02	-	-	63.03	-
Overall	100.0		36.44	35.97	27.90	100.0	41.78

Table 2. Table 2: Summary of analysis for the GBM data set (ZIP data set, resp.) with the Soft loss (the DWD loss, resp.) and a = a 1 𝑎 subscript 𝑎 1 a=a_{1} . The average proportion and misclassification or mis-refinement rate on each observation set p 1 subscript 𝑝 1 p_{1} , p 2 subscript 𝑝 2 p_{2} or p 3 subscript 𝑝 3 p_{3} for each of the regular classifier, classifier with rejection only and classifier with both reject and refine options are reported over 100 splits. A probability method for reject option is also compared. The overall empirical 0 0 - d 𝑑 d - 1 1 1 loss is reported for each classifier and each example. Proportions for selected prediction sets are reported as well. In the GBM example, { C , M } = { Classical , Mesenchymal } 𝐶 𝑀 Classical Mesenchymal \{C,M\}=\{\textrm{Classical},\textrm{Mesenchymal}\} and { N , P } = { Neural , Proneural } 𝑁 𝑃 Neural Proneural \{N,P\}=\{\textrm{Neural},\textrm{Proneural}\} . The results show improved overall loss by our methods and successful identification of a subset of most confusing classes.

GBM, Soft with $a=a_1$			Regular	Reject	R&R	Probability Method
	Proportion		Error			Proportion	Error
p1	72.15		13.69	13.69	13.69	72.58	13.78
p2	18.99	size 2: 17.14	41.35	39.53	3.724
		$\lfloor \{C,M\}$: 44.3%
		$\lfloor \{N,P\}$: 33.7%
		size 3: 1.853
p3	8.857		43.33	-	-	27.42	-
Overall	100.0		21.85	20.97	14.13	100.0	21.25
ZIP, DWD with $a=a_1$			Regular	Reject	R&R	Probability Method
	Proportion		Error			Proportion	Error
p1	97.05		2.087	2.087	2.087	98.90	2.607
p2	size 2: 2.578		35.71	28.57	0.110
	$\lfloor \{4,9\}$: 55.9%
p3	0.368		95.14	-	-	1.104	-
Overall	100.0		3.314	2.909	2.175	100.0	3.020

Equations

$$\displaystyle\mathcal{Y}_{j}=\left\{\begin{array}[]{ll}(k-1)^{-1/2}\boldsymbol{1}_{k-1}&~{}~{}~{}j=1,\\ -(1+k^{1/2})/\{(k-1)^{3/2}\}\boldsymbol{1}_{k-1}+\{k/(k-1)\}^{1/2}\boldsymbol{e}_{j-1}&~{}~{}~{}2\leq j\leq k,\end{array}\right.$$

$$\underset{\mathbf{f}\in \mathcal{F}}{\min }{n}^{-1}\sum _{i=1}^n\tau \{⟨{\mathcal{Y}}_{y_i},\mathbf{f}({\mathbf{x}}_i)⟩\},\text{subject to }J(\mathbf{f})\le s,$$

$$\displaystyle\sum_{j\neq y}\ell\{\langle\mathcal{Y}_{j},\boldsymbol{f}(\boldsymbol{x})\rangle\}=\sum_{j\neq y}\Big{[}\ell_{1}\{\langle\mathcal{Y}_{j},\boldsymbol{f}(\boldsymbol{x})\rangle\}+\ell_{2}\{\langle\mathcal{Y}_{j},\boldsymbol{f}(\boldsymbol{x})\rangle\}\Big{]},$$

$$\hat{\mathbf{f}}={\mathrm{argmin}}_{\mathbf{f}\in \mathcal{F}}{n}^{-1}\sum _{i=1}^n\sum _{j\ne y_i}\ell \{⟨{\mathcal{Y}}_j,\mathbf{f}({\mathbf{x}}_i)⟩\},\text{subject to }J(\mathbf{f})\le s.$$

$${\varphi }_{\hat{\mathbf{f}}}(\mathbf{x})=\{\begin{array}{ll}\text{mbox}\stackrel{◯}{R}& \text{if }{S}_{\delta }(⟨{\mathcal{Y}}_j,\hat{\mathbf{f}}⟩)=0,\forall j,\\ {\mathrm{argmax}}_{j=1,\dots ,k}⟨{\mathcal{Y}}_j,\hat{\mathbf{f}}(\mathbf{x})⟩& \text{otherwise}.\end{array}$$

$$\ell_{\textrm{SVM}}(u)=\left\{\begin{array}[]{ll}0&\textrm{if }u<-1,\\ 1+u&\textrm{if }-1\leq u<0,\\ 1+au&\textrm{otherwise},\end{array}\right.\makebox{and }\ell_{\textrm{DWD}}(u)=\left\{\begin{array}[]{ll}-\frac{1}{4u}&\textrm{if }u<-0.5,\\ 1+u&\textrm{if }-0.5\leq u<0,\\ 1+au&\textrm{otherwise}.\end{array}\right.$$

$$\text{allowdisplaybreaks}{\varphi }_{\hat{\mathbf{f}}}^{\text{set}}=\{\begin{array}{ll}\text{mbox}\stackrel{◯}{R}& \text{if }{S}_{\delta }(⟨{\mathcal{Y}}_j,\hat{\mathbf{f}}⟩)=0,\forall j,\\ \{j:S_{\delta }(⟨{\mathcal{Y}}_j,\hat{\mathbf{f}}⟩)>0\}& \text{if }{S}_{\delta }(⟨{\mathcal{Y}}_j,\hat{\mathbf{f}}⟩)>0,\text{for some }j,\\ \{j:S_{\delta }(⟨{\mathcal{Y}}_j,\hat{\mathbf{f}}⟩)=0\}& \text{otherwise}.\end{array}$$

$$\underset{{\alpha }_{ij},{\gamma }_{ij}}{\min }$$

$$0\le {\alpha }_{ij}\le A_{ij},0\le {\gamma }_{ij}\le A_{ij},i=1,\dots ,n,j=1,\dots ,k,$$

$$e(\mathbf{f},{\mathbf{f}}^{(p,k)})=\mathbb{E}[\sum _{j\ne Y}\ell \{⟨\mathbf{f}(\mathbf{X}),{\mathcal{Y}}_j⟩\}]-\mathbb{E}[\sum _{j\ne Y}\ell \{⟨{\mathbf{f}}^{(p,k)}(\mathbf{X}),{\mathcal{Y}}_j⟩\}].$$

$$\text{pr}\{|a(1-P_{(1)})-(1-P_{(k)})|<t\}\le c{t}^{\alpha }.$$

Full text

Keywords: Coordinate descent; Discriminant analysis; Diverging number of classes; High-dimensional data; Multi-class classification; Statistical learning theory.

1 Introduction

Classification is one of the founding pillars for statistical learning. In binary classification, an $i.i.d.$ training data set $\{(\boldsymbol{x}_{i},y_{i}),~{}i=1,\ldots,n\}$ is obtained from an unknown distribution $\mathbb{P}(\boldsymbol{x},y)$, where $\boldsymbol{x}\in\mathbb{R}^{p}$ is the observed covariates and $y\in\{+1,-1\}$ is the class label. The learning goal is to obtain a classifier $\phi(\cdot)$ based on the training data, such that for any new observation with only $\boldsymbol{x}$ available, its class label can be accurately predicted using $\phi(\boldsymbol{x})$. The goodness of a classifier is commonly measured by the misclassification rate, $\textrm{pr}\{\phi(\boldsymbol{X})\neq Y\}$, where the probability is taken with respect to $\mathbb{P}$. We aim to find the best classifier $\phi$ that minimizes the expected value of the [math]-$1$ loss $L(\boldsymbol{x},y,\phi)=\mathbbm{1}_{\left\{{\phi(\boldsymbol{x})\neq y}\right\}}$.

There are many classification methods in the literature. For an overall introduction, see Hastie et al. (2009). Among these methods, margin-based classifiers are very popular. For a binary margin-based classifier, one typically finds a classification function $f:\mathbb{R}^{p}\rightarrow\mathbb{R}$ and defines the classifier as $\phi(\boldsymbol{x})=\textrm{sign}\{f(\boldsymbol{x})\}$. A correct classification occurs when the functional margin $yf(\boldsymbol{x})>0$. Since directly minimizing the empirical [math]-$1$ loss is difficult due to the discontinuity of the [math]-$1$ loss function, a surrogate loss is often used to encourage large values of the functional margin $yf(\boldsymbol{x})$. Many binary margin-based classifiers using different surrogate loss functions have been proposed in the literature, such as Support Vector Machines (SVM; Cortes and Vapnik, 1995; Vapnik, 1998), AdaBoost (Freund and Schapire, 1997), $\psi$-learning (Shen et al., 2003), Distance-Weighted Discrimination (DWD; Marron et al., 2007), Large-margin Unified Machine (LUM; Liu et al., 2011) and Flexible High-dimensional Classification Machines (FLAME; Qiao and Zhang, 2015).

When there are $k>2$ classes, the class label $y$ can be coded as $y\in\{1,\ldots,k\}$ instead. In this article, we focus on multicategory classifiers that consider all classes simultaneously in a single optimization problem. A common approach is to train a vector-valued function $\boldsymbol{f}=(f_{1},\ldots,f_{k})^{T}:\mathbb{R}^{p}\mapsto\mathbb{R}^{k}$, and define the classifier as $\phi(\boldsymbol{x})=\mathop{\rm argmax}_{j\in\{1,\ldots,k\}}f_{j}(\boldsymbol{x})$. A sum-to-zero constraint, $\sum_{j=1}^{k}f_{j}\equiv 0$, is often imposed for theoretical and practical concerns. See, for example, Vapnik (1998), Crammer and Singer (2001), Lee et al. (2004), Zhu and Hastie (2005), Liu and Shen (2006), Liu and Yuan (2011), Zhang and Liu (2013), among others. Recently, Zhang and Liu (2014) proposed the angle-based classification framework. The angle-based classifiers are free of the sum-to-zero constraint, and can be advantageous in terms of computational speed and classification performance, especially for high-dimensional problems. In this paper, our proposed method is based on the angle-based classification framework.

In real applications, it is often the case that an accurate decision is hard to reach, and the consequence of misclassification is disastrous and too severe to bear. In these situations, it may be wise to resort for a reject option, i.e., to report “I don’t know” (denoted as ® hereafter), to avoid such a consequence. With a reject option, future resources will be allocated to these previously rejected subjects to improve their classification. For example, in cancer diagnosis, an oncologist should send a patient, who is difficult to be diagnosed based on preliminary results, for more tests, or seek a second opinion, instead of telling the patient, with little confidence, that she probably has or does not have the cancer.

To adopt a reject option, a possible approach is to modify the [math]-$1$ loss such that when ® occurs, a positive cost is present (otherwise, ® would always be preferred). For instance, Herbei and Wegkamp (2006) considered the [math]-$d$-$1$ loss, $L(\boldsymbol{x},y,\phi)=d\cdot\mathbbm{1}_{\left[{\phi(\boldsymbol{x})=\mbox{\textregistered}}\right]}+\mathbbm{1}_{\left[{\phi(\boldsymbol{x})\neq y,\phi(\boldsymbol{x})\neq\mbox{\textregistered}}\right]}$, where $d>0$ is the cost for a rejection (e.g., this may be the cost for the additional tests that the oncologist orders for the patient.)

Recently, there have been a number of works on the reject option for binary classification in the literature (Fumera and Roli, 2002; Herbei and Wegkamp, 2006; Wegkamp, 2007; El-Yaniv and Wiener, 2010; Yuan and Wegkamp, 2010; Wegkamp and Yuan, 2011). However, much less attention has been paid to multicategory classification. In the literature, Fumera et al. (2000), Tax and Duin (2008) and Le Capitaine and Frélicot (2010) considered the reject option in multicategory classification using methods that depend on explicit class conditional probability estimation. However, probability or density estimation is often much more difficult than class label prediction (Fürnkranz and Hüllermeier, 2010), especially when the dimension is high (Zhang et al., 2013). Hence, it is desirable to have a multicategory classifier with a reject option that does not rely on explicit class probability estimation. The current article fills the gap on this end.

Our first contribution is to propose multicategory classifiers with a reject option. Our methods are based on angle-based multicategory methods and do not involve estimating the class conditional probability, hence can be robust and efficient for high-dimensional problems.

Secondly, we introduce a new notion that is quite unique for the multicategory problem (which is absent in the binary case), namely, a refine option. A refinement predicts the class label to be from a set of $r$ labels, where $1\leq r\leq k$. When $r=1$, it reduces to the regular definite classification; when $r=k$, no information is provided and a refinement is the same as ®; when $1<r<k$, we have refined the number of classes that an observation most likely belongs to, from $k$ to $r$. A smaller $r$ leads to more useful information, yet it increases the chance of misclassification. In this paper, we introduce a data-adaptive approach that can automatically select the size $r$ for a new prediction.

The usefulness of the refine option can be understood from two sides. In contrast to a definite but potentially reckless answer ($r=1$), a refinement is more cautious and risk-avert; catastrophic consequences of misclassification can be effectively avoided. On the other hand, compared with a complete reject option ($r=k$), which tells little about an observation, a refinement provides constructive information; future investigation can be conducted on a set of originally confusable classes, which can improve the classification performance.

Our next contribution is a thorough investigation of the theoretical properties of our methods, focusing on the asymptotic behavior of the excess $\ell$-risk when the number of classes $k$ and the dimension $p$ both diverge. In particular, we calibrate the difficulty of classification when $k$ increases. This helps to shed some light on the usefulness of our new refine option, that is, one can focus on a subset of classes in a refined further analysis, which can in turn improve the classification accuracy. Moreover, we demonstrate that if the number of noise predictors diverges faster than $k$ does, then the $L_{1}$ penalty can perform better than the $L_{2}$ regularization. On the other hand, if the number of noise predictors is negligible with respect to the number of classes, then the $L_{1}$ and $L_{2}$ methods are comparable.

The rest of the article is organized as follows. Section 2 provides some background information. The main methods are introduced in Section 3. Section 4 presents the algorithms and tuning parameter selection. A novel statistical learning theory is provided in Section 5. Section 6 includes all the numerical studies. Some concluding remarks are given in Section 7. Most technical proofs are collected in the Supplementary Materials.

2 Background

Let $P_{j}(\boldsymbol{x})=\textrm{pr}(Y=j\mid\boldsymbol{X}=\boldsymbol{x})$ be the class conditional probability of observation $\boldsymbol{x}$ for class $j$ ($j=\pm 1$ or $1,\dots,k$). In the binary case, it can be shown that the Bayes decision under the [math]-$d$-$1$ loss is, $\phi_{Bayes}(\boldsymbol{x})=+1$ if $P_{+1}(\boldsymbol{x})\geq 1-d$, $-1$ if $P_{-1}(\boldsymbol{x})\geq 1-d$, or ® otherwise (Herbei and Wegkamp, 2006).

Note that $\sum_{j}P_{j}(\boldsymbol{x})=1$ where $j=\pm 1$ for binary classification or $1,\dots,k$ for the multicategory case. Hence, for each $\boldsymbol{x}$, $(P_{j}):=(P_{j}(\boldsymbol{x}))_{j=\pm 1\textrm{ or }1,\dots,k}$ must fall on a simplex in $\mathbb{R}^{k}$. Throughout this article, we define the Bayes reject region to be $R_{\textrm{Bayes}}:=\{(P_{j}):\phi_{Bayes}(\boldsymbol{x})=\mbox{\textregistered}\}$, a region on this simplex. For example, in the binary case, we have $R_{\textrm{Bayes}}=\{(P_{+1},P_{-1}):d<P_{+1}(\boldsymbol{x})<1-d\}$.

While it is possible to achieve the reject option by first estimating the conditional probabilities $P_{j}(\boldsymbol{x})$ for each $\boldsymbol{x}$ and then plugging the estimates in the Bayes rule (whose form in the multicategory case will be formally presented in Proposition 2), it is well known that probability estimation can be more difficult than mere label prediction (Wang et al., 2008; Fürnkranz and Hüllermeier, 2010; Wu et al., 2010), especially when the dimension $p$ is large (Zhang et al., 2013). Hence our goal here is to propose multicategory classifiers with a reject option that does not require explicit probability estimation.

We first briefly introduce the state-of-the-art for binary classification with a reject option. Section 2.2 reviews the angle-based multicategory classification methods.

2.1 Binary Margin-based Classification with a Reject Option

The seminal paper of Bartlett and Wegkamp (2008) proposed a novel method that employed a modified hinge loss $\psi$ for binary classification with a reject option. In particular, $\psi(u)=0$ if $u\geq 1$, $\psi(u)=1-u$ if $0\leq u<1$, and $\psi(u)=1-au$ otherwise, where $a=(1-d)/d>1$ (see Figure 1.) Define $f^{*}(\boldsymbol{x})=\mathop{\rm arginf}_{f\in\mathcal{F}}\mathbb{E}[\psi\{Yf(\boldsymbol{X})\}\mid\boldsymbol{X}=\boldsymbol{x}]$ to be the minimizer of the conditional expected loss (for an appropriate space $\mathcal{F}$) and define the associated classifier to be $\phi_{f^{*}}(\boldsymbol{x})=\mbox{\textregistered}$ if $f^{*}(\boldsymbol{x})=0$, or $\textrm{sign}\{f^{*}(\boldsymbol{x})\}$ otherwise. Then the $f^{*}$-reject region is defined as $R_{f^{*}}=\{(P_{+1},P_{-1}):f^{*}(\boldsymbol{x})=0\}$. Bartlett and Wegkamp (2008) showed that their $\phi_{f^{*}}$ coincided with the Bayes rule $\phi_{Bayes}$ and hence, $R_{\textrm{Bayes}}=R_{f^{*}}$.

2.2 Angle-based Multicategory Classification

Zhang and Liu (2014) showed that multicategory margin-based classification methods with $k$ classification functions under the sum-to-zero constraint can be inefficient, and proposed the angle-based classification framework. They showed that angle-based classifiers are competitive in terms of classification accuracy and computational speed, especially when $p$ is large. The idea of angle-based classifiers are briefly introduced here. For a problem with $k$ classes, consider a centered simplex in $\mathbb{R}^{k-1}$ with $k$ vertices, $\mathcal{Y}=\{\mathcal{Y}_{1},\ldots,\mathcal{Y}_{k}\}$. Here

[TABLE]

where $\boldsymbol{1}_{k-1}\in\mathbb{R}^{k-1}$ is a vector of all $1$’s, and $\boldsymbol{e}_{j}\in\mathbb{R}^{k-1}$ has $1$ on its $j$th element and [math] elsewhere. One can verify that $\mathcal{Y}_{j}$’s have unit norms, and the pairwise distances between $\mathcal{Y}_{i}$ and $\mathcal{Y}_{j}$ are the same for all $i\neq j\in\{1,\ldots,k\}$. Therefore, $\mathcal{Y}$ forms a simplex with $k$ vertices in $\mathbb{R}^{k-1}$. We use $\mathcal{Y}_{j}$ as the surrogate coding vector for the class label ‘$j$’. In angle-based methods, a vector-valued classification function $\boldsymbol{f}$ maps $\boldsymbol{x}$ to $\boldsymbol{f}(\boldsymbol{x})\in\mathbb{R}^{k-1}$. Each $\boldsymbol{f}(\boldsymbol{x})$ induces $k$ angles with $\mathcal{Y}_{1},\ldots,\mathcal{Y}_{k}$, namely, $\angle(\mathcal{Y}_{j},\boldsymbol{f})$, $j=1,\ldots,k$. Zhang and Liu (2014) proposed to use the prediction rule $\phi(\boldsymbol{x})=\mathop{\rm argmin}_{j=1,\ldots,k}\angle(\mathcal{Y}_{j},\boldsymbol{f}(\boldsymbol{x}))=\mathop{\rm argmax}_{j=1,\ldots,k}\langle\mathcal{Y}_{j},\boldsymbol{f}(\boldsymbol{x})\rangle$. Here, the inner product $\langle\mathcal{Y}_{j},\boldsymbol{f}(\boldsymbol{x})\rangle$ can be viewed as an analog to the functional margin in a non-angle-based method, and hence is referred to as an angle margin hereafter. From this point of view, Zhang and Liu (2014) proposed to solve the following optimization problem to find $\boldsymbol{f}$ within some functional space $\mathcal{F}$,

[TABLE]

where $\tau(\cdot)$ is a common binary margin-based surrogate loss function, $J(\boldsymbol{f})$ is a penalty on $\boldsymbol{f}$ to prevent overfitting, and $s$ is a tuning parameter to balance the goodness of fit and the complexity of the model. The optimization (2) encourages a large value for $\langle\mathcal{Y}_{y_{i}},\boldsymbol{f}(\boldsymbol{x}_{i})\rangle$.

3 Methodology

In this section, we introduce our main methods, namely, a multicategory classifier with a reject option in Section 3.1, and one with both reject and refine options in Section 3.2.

3.1 Multicategory Classification with a Reject Option

Given an observation $\boldsymbol{x}$, recall the definition of $P_{j}(\boldsymbol{x})$. Let $P_{(j)}(\boldsymbol{x})$ be the $j$th greatest value among $P_{j}(\boldsymbol{x})$’s, let $y_{(j)}$ be the class label corresponding to $P_{(j)}(\boldsymbol{x})$, and define $\mathcal{Y}_{(j)}$ to be the coding vector for $y_{(j)}$. Note that $y_{(j)}$ is not necessarily the true class label for $\boldsymbol{x}$, but is its $j$th most plausible class. Lastly, we define $Q_{j}=1-P_{j}$, and $Q_{(j)}=1-P_{(j)}$.

Our approach is inspired by the work of Bartlett and Wegkamp (2008) for binary problems. In particular, their loss function was $\psi(u)=H(u)+(a-1)[-u]_{+}$, where $H(u):=[1-u]_{+}$ was the hinge loss function for SVM and $a-1=(1-d)/d-1>0$ was an additional slope added to the hinge loss for $u<0$. One can view $\psi$ as the hinge loss, bent at $u=0$ so that the left derivative $-a$ is (negatively) larger than the right derivative $-1$. Denote the theoretical minimizer $f^{*}(\boldsymbol{x})=\mathop{\rm arginf}_{f\in\mathcal{F}}\mathbb{E}[\psi\{Yf(\boldsymbol{X})\}\mid\boldsymbol{X}=\boldsymbol{x}]$. The bent loss function can keep $f^{*}$ at [math] if $P_{j}$ is not significantly different from $1-P_{j}$ ($j=+1,-1$), thus leading to a rejection in this case. In particular, $f^{*}$ is positive if $P_{+1}>1-d$, is negative if $P_{+1}<d$, and remains [math] if $d<P_{+1}<1-d$. Note that comparing $P_{+1}$ with $\{d,1-d\}$ is equivalent to comparing $Q_{+1}/Q_{-1}$ with $\{1/a,a\}$.

Inspired by these observations, to realize a reject option for multicategory classification, we employ a similar technique, namely, to use a bent loss function that has different left and right derivatives at [math]. Specifically, we equip an angle-based multicategory classifier with a modified loss, with the aim to have the angle margin $\langle\mathcal{Y}_{j},\boldsymbol{f}^{*}(\boldsymbol{x})\rangle=0$ for all $j=1,\ldots,k$, where $\boldsymbol{f}^{*}(\boldsymbol{x})$ is the theoretical minimizer of the loss (to be defined more precisely later) if the class conditional probability $P_{j}(\boldsymbol{x})$’s are not significantly different from each other; note that this implies that $P_{(1)}$ is not large enough and that $Q_{j}$’s are similar as well. We will show in Proposition 1 that this is indeed the case.

For any observation $(\boldsymbol{x},y)$ and function $\boldsymbol{f}$, we propose a loss function defined as

[TABLE]

where $\ell(u)=\ell_{1}(u)+\ell_{2}(u)$. Here $\ell_{1}(u)=\tau(-u)$ and $\tau$ is the loss function for any Fisher consistent binary margin-based classifier (such as the hinge loss, the DWD loss, the LUM loss and the FLAME loss.) Throughout this paper we assume $\ell_{1}^{\prime}(0)=1$ for simplicity. Furthermore, $\ell_{2}(u)$ is defined so that $\ell^{\prime}(u)\equiv a>1$ for $u>0$, and $\ell_{2}(u)=0$ for $u<0$. Hence $\ell$ is the result of bending $\ell_{1}$ using $\ell_{2}$. This will be illustrated in Figure 2 using two typical loss functions. The loss function (3) is the sum of $\ell$ over all class $j$’s not equal to the true class $y$. With this loss function, our classification function is obtained by,

[TABLE]

The monotonically increasing loss function $\ell$ encourages a small value of $\langle\mathcal{Y}_{j},\boldsymbol{f}(\boldsymbol{x}_{i})\rangle$ for $j\neq y_{i}$ which indirectly maximizes $\langle\mathcal{Y}_{y_{i}},\boldsymbol{f}(\boldsymbol{x}_{i})\rangle$ since $\sum_{j=1}^{k}\mathcal{Y}_{j}=\boldsymbol{0}$.

With $\delta$ a small positive constant, define the soft thresholding operator (Donoho, 1995) as $S_{\delta}(c)=\textrm{sign}(c)\max(|c|-\delta,0)$. The induced classifier can be summarized as,

[TABLE]

That is, we report a rejection when all $\langle\mathcal{Y}_{j},\hat{\boldsymbol{f}}\rangle$’s are close to 0.

Our method is very general, as one can use any Fisher consistent binary margin-based loss and extend the binary classifier to the multicategory case, meanwhile allowing for a reject option. For the purpose of illustration, in this section we generalize two popular binary margin-based classifiers, SVM and DWD. The bent SVM and DWD losses are,

[TABLE]

We plot $\ell_{\textrm{SVM}}$ and $\ell_{\textrm{DWD}}$ in Figure 2.

To provide more insights to the new classifier, we first study the population version of $\hat{\boldsymbol{f}}$, namely, the theoretical minimizer $\boldsymbol{f}^{*}$, and its associated reject region. We will compare the reject region of our method with the Bayes reject region under a generalized [math]-$d$-$1$ loss, and show that our methods mimic the latter, which helps to justify our approach from a theoretical view.

Proposition 1**.**

Let $\ell$ be a bent loss function as defined in (3), with $\ell^{\prime}(0-)=1$ and $\ell^{\prime}(0+)=a>1$. For the sequence $Q_{(1)}\leq Q_{(2)}\leq\dots\leq Q_{(k)}$, if there exists some $1\leq s\leq k-1$ such that $Q_{(s)}/Q_{(1)}<a$ and $Q_{(s+1)}/Q_{(1)}\geq a$, then the theoretical minimizer $\boldsymbol{f}^{*}$ of the conditional expected loss $\mathbb{E}\{\sum_{j\neq Y}\ell\{\langle\mathcal{Y}_{j},\boldsymbol{f}(\boldsymbol{X})\rangle\}\mid\boldsymbol{X}=\boldsymbol{x}\}$ satisfies that $\langle\mathcal{Y}_{(1)},\boldsymbol{f}^{*}(\boldsymbol{x})\rangle>0$, $\langle\mathcal{Y}_{(2)},\boldsymbol{f}^{*}(\boldsymbol{x})\rangle=\dots=\langle\mathcal{Y}_{(s)},\boldsymbol{f}^{*}(\boldsymbol{x})\rangle=0$, and $\langle\mathcal{Y}_{(t)},\boldsymbol{f}^{*}(\boldsymbol{x})\rangle<0$ for all $t\geq s+1$; otherwise, $\langle\mathcal{Y}_{(j)},\boldsymbol{f}^{*}(\boldsymbol{x})\rangle=0$ for all $j=1,\dots,k$.

Proposition 1 indicates that $\langle\mathcal{Y}_{(j)},\boldsymbol{f}^{*}(\boldsymbol{x})\rangle=0$ for all $j$ when $Q_{(k)}/Q_{(1)}<a$, that is, the class conditional probability of the most plausible class $y_{(1)}$ is not significantly different from that of the least plausible class $y_{(k)}$, by a ratio not exceeding $a>1$.

Hence, the corresponding $\boldsymbol{f}^{*}$-reject region is $R_{\boldsymbol{f}^{*}}(a)=\{(P_{1},\ldots,P_{k}):\langle\mathcal{Y}_{j},\boldsymbol{f}^{*}\rangle=0,~{}\forall j\}=\{(P_{1},\ldots,P_{k}):Q_{(k)}<aQ_{(1)}\}$ which depends on the parameter $a$. When the context is clear, we may use notation $R_{\boldsymbol{f}^{*}}$ without explicitly declaring its dependence on $a$. On Panels (b) and (c) of Figure 3, we plot $R_{\boldsymbol{f}^{*}}(a)$ for a three-class example with two values of $a$, $a_{1}$ and $a_{2}$, defined in Proposition 3.

For each $a$, the $\boldsymbol{f}^{*}$-reject region is near the center of the simplex, which is where the class conditional probability $P_{j}$’s are close to each other. Intuitively, that is a difficult observation to classify. Next, consider a natural generalization of the (binary) [math]-$d$-$1$ loss in Herbei and Wegkamp (2006) to the multicategory case, which assigns [math] for correct decisions, $1$ for mistakes, and $d$ for ®. In a $k$-class problem, we must have $0<d\leq(k-1)/k$ to prevent the reject option from being inadmissible. The next proposition gives the Bayes classifier under the generalized [math]-$d$-$1$ loss for multicategory classification, which depends on $P_{(1)}$ only. The Bayes reject region is $R_{\textrm{Bayes}}=\{(P_{1},\ldots,P_{k}):P_{(1)}\leq 1-d\}$ (see Panel (a) of Figure 3.)

Proposition 2**.**

(Chow, 1970)* For the [math]-$d$-$1$ loss in multicategory classification, the Bayes classifier is $\phi_{\textrm{Bayes}}(\boldsymbol{x})=y_{(1)}$ if $P_{(1)}(\boldsymbol{x})>1-d$, and ® otherwise.*

One would expect a good classifier with a reject option to have a reject region that resembles (or even coincides with) that of the Bayes rule (under an appropriate loss function). Indeed, one can deduct from Proposition 1 that for any Fisher consistent binary loss function with $k=2$ and $a=(1-d)/d$, our $R_{\boldsymbol{f}^{*}}$ coincides with the Bayes reject region $R_{\textrm{Bayes}}$ under the [math]-$d$-$1$ loss. However, in the multicategory case, this property generally does not hold. The next proposition gives the greatest $a_{1}$ and smallest $a_{2}$ such that $R_{\boldsymbol{f}^{*}}(a_{1})$ and $R_{\boldsymbol{f}^{*}}(a_{2})$ bound $R_{\textrm{Bayes}}$ from two sides.

Proposition 3**.**

For a $k$-class problem with the cost for rejection $d$, define $a_{1}=(k-1-d)/(kd-d)$ and $a_{2}=(k-1)(1-d)/d$. Then we have $R_{\boldsymbol{f}^{*}}(a_{1})\subset R_{\textrm{Bayes}}\subset R_{\boldsymbol{f}^{*}}(a_{2})$. The bounds are tight in the sense that for any $a$ such that $a_{1}<a<a_{2}$, $R_{\boldsymbol{f}^{*}}(a)\not\subset R_{\textrm{Bayes}}$ and $R_{\textrm{Bayes}}\not\subset R_{\boldsymbol{f}^{*}}(a)$.

Panels (b) and (c) in Figure 3 show the $\boldsymbol{f}^{*}$-reject regions for $a_{1}$ and $a_{2}$. From the comparison between these two reject regions and the Bayes reject region shown in Panel (a), one can see that our method induces a reject region that closely approximates the Bayes reject region. In practice, one can choose $a$ from $[a_{1},a_{2}]$ for such an approximation. The issue of tuning the parameter $a$ is deferred to Section 4.2.

In each panel among (a), (b) and (c), the reject region occupies the center of the simplex where all $P_{j}$ are close to each other (i.e. $P_{(1)}$ is not large enough). Out of that area, some or all the classes other than the dominating class $y_{(1)}$ would appear to be unlikely and hence are ruled out. In this case, a rejection is not yielded by (5).

3.2 Classification with a Refinement Option

The previous subsection is built on the assumption that a reject option is necessary when an observation falls into the reject region, depicted in Figure 3, where all classes seem to be equally likely and it is difficult to distinguish one class from another. On the other hand, even if an observation is not in the reject region, it is not necessarily the case that a definite classification is desirable. This is the main point of the current subsection. In each of Panels (a)-(c) of Figure 3, out of the blue reject region, there are still areas where some confusion may occur between two classes. For example, many observations near the boundary between the black (class 1) and the red (class 2) regions are not likely to be from class 3, but we still have difficulty determining between class 1 and class 2. A method which is only capable of yielding rejections is still not able to effectively avoid an expensive misclassification which is very likely to happen in this situation. This naturally motivates a new refine option for multicategory classification, in which, we may rule out class 3 and predict the observation to be from either class 1 or class 2. On one hand, we can avoid a potential misclassification by using a set of classes as the prediction; on the other hand, the set prediction provides additional information compared to what a rejection would do (which is almost null.)

The discussion above suggests that the complement of the reject region (the previous definite regions) be further partitioned to some definite regions and refine regions. In Figure 3, for example, in addition to rejections, we should have (a) three definite regions where the prediction is a single class label, 1, 2 or 3, and (b) three refine regions where the prediction is a set of two classes, namely, $\{1,2\}$, $\{2,3\}$ or $\{1,3\}$.

To this end, we review the results of Proposition 1: a rejection occurs (all the angle margins $\langle\mathcal{Y}_{(s)},\boldsymbol{f}^{*}(\boldsymbol{x})\rangle=0$) when the most plausible class $y_{(1)}$ cannot be distinguished from the least plausible one $y_{(k)}$ (since $Q_{(k)}<aQ_{(1)}$); otherwise, the angle margin $\langle\mathcal{Y}_{(1)},\boldsymbol{f}^{*}(\boldsymbol{x})\rangle$ for the most plausible class $y_{(1)}$ is positive, the angle margins for some less plausible classes $s$ are zero, although the conditional probabilities of these classes are still close to that of $y_{(1)}$ (since $Q_{s}<aQ_{(1)}$), and the angle margins for the implausible classes are all negative. Hence we may use the angle margins to define predictions, since they reflect the plausibility of a class label for an observation. A general guideline is that a positively large angle margin suggests a label prediction, the presence of some angle margins close to 0 and some angle margins negatively large suggests refinement (and ruling out those implausible), and the case of all angle margin close to 0 indicates rejection.

In reality, since the empirical angle margin $\langle\mathcal{Y}_{j},\hat{\boldsymbol{f}}(\boldsymbol{x})\rangle$ may be deviated from the theoretical counterpart $\langle\mathcal{Y}_{j},\boldsymbol{f}^{*}(\boldsymbol{x})\rangle$ for a finite sample problem, the gap between angle margins may not appear obvious. In this case, we employ a soft-thresholding technique to distinguish significantly large and small angle margins. In particular, with the thresholded angle margins, our new classifier with both reject and refine options is defined as,

[TABLE]

Note that the reject rule, the first line in (6), is identical to that in (5). This corresponds to the case that all angle margins are close to 0, implying that all the class conditional probabilities are close to each other. The second line attempts to find the most significantly large margin, and hence the most plausible class. In our numerical experience, we occasionally observe cases with multiple significantly large margins which are close to each other. In this case, we have chosen to include all those plausible classes (if any) as a set prediction. The third line corresponds to the case where the most plausible class is not significantly different from some other classes and we resort to ruling out those implausible classes (those with significantly negatively large margins) instead.

It can be seen that the union of the second and third cases in (6) is identical to the definite label prediction region in (5) (the second line therein). However, when an observation belongs to the third case in (6), the rule in (5) recklessly reports a single label as the prediction, while the novel refinement rule (6) here uses a set prediction. This is the main difference between the classifiers in (5) and (6).

For illustration, we plot the reject, refine and definite regions for a three-class problem on Panel (d) of Figure 3 for $a_{2}$. It can be seen that the three refine (cyan) regions are cut from the previous definite regions in Panel (c) and hence the current definite regions are smaller than in (c) as well. More importantly, one may hold more confidence for a label prediction made by the new classifier (6). In Section 6, we demonstrate through numerical examples that the classification accuracy on the refine region in (6) can be significantly improved, compared to the classification accuracy on the counterpart of (5).

In both (5) and (6), the choice of $\delta$ is a matter of tuning parameter. The details of tuning $\delta$ are given in the next section.

4 Optimization and Tuning Parameter Selection

In this section, we discuss how to solve the optimization problem (4), from which both our methods (5) and (6) are derived. Various approaches are possible, depending on the choice of $\ell$, $\mathcal{F}$ and $J(\boldsymbol{f})$. For demonstration purpose, in this section we use the reversed hinge loss for $\ell_{1}$. We let $J(\boldsymbol{f})$ be the $L_{2}$ norm penalty in linear learning, and the squared norm penalty in kernel learning (Schölkopf and Smola, 2002; Shawe-Taylor and Cristianini, 2004). For other cases with some general properties, such as one with a differentiable $\ell_{1}$ and a separable penalty function, one can solve (4) by the alternating direction method of multipliers (Boyd et al., 2011). We have developed fast implementations for our methods based on the hinge loss, the DWD loss and the Soft classifier loss (Liu et al., 2011). These algorithms will be publicly available in R.

4.1 Optimization

We start our discussion from linear learning. Suppose $f_{q}(\boldsymbol{x})=\boldsymbol{x}^{T}\boldsymbol{\beta}_{q}$ for $q=1,\ldots,k-1$. Notice that we include the intercept terms in the $\boldsymbol{\beta}_{q}$’s by catenating $1$ to $\boldsymbol{x}$. The $L_{2}$ penalty $J(\boldsymbol{f})$ can be written as $J(\boldsymbol{f})=\sum_{q=1}^{k-1}\boldsymbol{\beta}_{q}^{T}\boldsymbol{\beta}_{q}$. The bent hinge loss $\ell$ can be decomposed as $\ell(u)=[1+u]_{+}+(a-1)[u]_{+}$. After a series of introduction of Lagrangian multiplier and slack variables, and manipulations due to the KKT conditions (detailed derivations of the algorithms can be found in the Supplementary Materials), we can show that the optimization problem (4) is equivalent to

[TABLE]

where $\boldsymbol{\beta}_{q}=-\frac{1}{n\lambda}\sum_{i=1}^{n}\sum_{j\neq y_{i}}\{\alpha_{ij}+(a-1)\gamma_{ij}\}\mathcal{Y}_{j,q}\boldsymbol{x}_{i}.$ Observe that the objective function is quadratic in terms of $\alpha_{ij}$’s and $\gamma_{ij}$’s, and the constraints are box constraints. Therefore, one can solve (4.1) via the very fast coordinate descent algorithm (Friedman et al., 2010). Moreover, as the objective function is quadratic, for each coordinate-wise update, the solution can be explicitly calculated. This greatly boosts the computational speed.

Similarly, for kernel learning, we can use $f_{q}(\boldsymbol{x})=\sum_{i=1}^{n}K(\boldsymbol{x}_{i},\boldsymbol{x})\theta_{q,i}+\theta_{q,0}$, $q=1,\ldots,k-1$, for kernel function $K(\cdot,\cdot)$, where the square norm penalty is $\sum_{q=1}^{k-1}\boldsymbol{\theta}_{q}^{T}\boldsymbol{K}\boldsymbol{\theta}_{q}$, and $\theta_{q,i}$ is the $i$th element of $\boldsymbol{\theta}_{q}$. In the same manner as above, one can derive a fast solution to this problem.

4.2 Tuning Parameter Selection

There are three tuning parameters in our methods, namely $a$, $s$ and $\delta$. Here $a$ is associated with the cost of rejection $d$, where the latter should be fixed a priori. In the numerical study, we find that the choice of $a$ does not affect the result much, as long as $a_{1}<a<a_{2}$. We recommend to try both $a_{1}$ and $a_{2}$ and use the one with a better result.

Parameter $s$ restricts the model space that the classifier is searched from. Typically $s$ is tuned from a grid of many candidate values. The optimal $s$ is chosen for one that minimizes the [math]-$d$-$1$ loss for a separate tuning data set or via cross-validation.

Lastly, $\delta>0$ is a small positive constant used to distinguish significantly large and small angle margins. Similar to $s$, we tune $\delta$ by choosing the one that leads to the smallest [math]-$d$-$1$ loss for a separate tuning data set or via cross-validation. However, note that solving the optimization problem (4) to obtain $\hat{\boldsymbol{f}}$ does not involve $\delta$; only the conversion from $\hat{\boldsymbol{f}}$ to the classifier $\phi_{\hat{\boldsymbol{f}}}$ or $\phi_{\hat{\boldsymbol{f}}}^{\textrm{set}}$ does. Hence tuning $\delta$ hardly adds to the computational cost.

5 Statistical Learning Theory

In this section, we first study the convergence rate of the excess $\ell$-risk under various settings. In particular, we study the cases of linear learning with $L_{1}$ and $L_{2}$ penalties, and kernel learning with the squared norm penalty. Then, we improve our results with an additional low noise assumption, analogous to Tsybakov’s margin condition (Tsybakov, 2004).

5.1 General Convergence Rate of the Excess $\ell$-Risk

In the literature, the excess $\ell$-risk for a learning procedure has been studied by many authors in different settings. See Zhang (2004) and Bartlett et al. (2006) for standard binary classification, Liu and Shen (2006), Wang and Shen (2007), and Zhang and Liu (2014) for multicategory classification, and Herbei and Wegkamp (2006), Wegkamp (2007), and Wegkamp and Yuan (2011) for binary classification with a reject option. We focus on the excess $\ell$-risk for the multicategory classification with a reject option.

We first consider linear learning with a diverging number of predictors $p$ and a diverging number of classes $k$. In the statistical learning literature, it is becoming increasingly popular to consider large $p$ as $n\rightarrow\infty$ (for example, Fan and Lü, 2008; Mai and Zou, 2012; Cai et al., 2014, among others.) On the other hand, for classification problems, not much attention has been paid to the large $k$ situation. Recently, Gupta et al. (2014) studied classification problems with tens of thousands of classes. However, the theoretical property of classifiers with diverging $k$ remains largely unknown.

First, we assume that each predictor is bounded within $[0,1]$, though our theory can be generalized to cases where it is uniformly bounded. As the number of predictors $p$ and the number of classes $k$ diverge, we let the underlying distribution $\mathbb{P}(\boldsymbol{x},y)$ be defined on \big{(}[0,1]^{\infty}\times\{1,\ldots,k,\ldots,\},\sigma^{\infty}([0,1]^{\infty})\times 2^{\{1,\ldots,k,\ldots,\}}\big{)}, where $\sigma^{\infty}([0,1]^{\infty})$ is the $\sigma$-field generated by open balls with the topology under the uniform metric $d(\boldsymbol{x},\boldsymbol{x}^{\prime})=\sup_{l=1,\dots,k,\dots}|x_{l}-x^{\prime}_{l}|$, and $2^{\{1,\ldots,k,\ldots,\}}$ is the power set of $\{1,\ldots,k,\ldots,\}$ and hence a $\sigma$-field.

For linear learning, we have $\boldsymbol{f}=(f_{1},\ldots,f_{k-1})^{T}$ with $f_{q}(\boldsymbol{x})=\boldsymbol{\beta}_{q}^{T}\boldsymbol{x}$, $q=1,\ldots,k-1$. We define $\mathcal{F}(p,k,s)=\{\boldsymbol{f}=(f_{1},\ldots,f_{k-1})^{T}:f_{q}(\boldsymbol{x})=\boldsymbol{\beta}_{q}^{T}\boldsymbol{x},~{}q=1,\ldots,k-1,~{}J(\boldsymbol{f})\leq s\}$. For the $L_{1}$ penalty, $J(\boldsymbol{f})=\sum_{q=1}^{k-1}\|\boldsymbol{\beta}_{q}\|_{1}$, and for the $L_{2}$ penalty, $J(\boldsymbol{f})=\sum_{q=1}^{k-1}\|\boldsymbol{\beta}_{q}\|_{2}^{2}$. Let $\mathcal{F}(p,k)=\bigcup_{0\leq s<\infty}\mathcal{F}(p,k,s)$ be the full $p$-dimensional model with $k$ classes. Recall that $\hat{\boldsymbol{f}}=\mathop{\rm argmin}_{\boldsymbol{f}\in\mathcal{F}(p,k,s)}\frac{1}{n}\sum_{i=1}^{n}\sum_{j\neq y_{i}}\ell\{\langle\boldsymbol{f}(\boldsymbol{x}_{i}),\mathcal{Y}_{j}\rangle\}.$ Let the best classification function be denoted by $\boldsymbol{f}^{(p,k)}=\mathop{\rm arginf}_{\boldsymbol{f}\in\mathcal{F}(p,k)}\mathbb{E}[\sum_{j\neq y}\ell\{\langle\boldsymbol{f}(\boldsymbol{x}),\mathcal{Y}_{j}\rangle\}]$.

For any classification function $\boldsymbol{f}$, the excess $\ell$-risk $e(\boldsymbol{f},\boldsymbol{f}^{(p,k)})$ is defined as

[TABLE]

We denote $d_{n,p,k}=\inf_{\boldsymbol{f}\in\mathcal{F}(p,k,s)}e_{\ell}(\boldsymbol{f},\boldsymbol{f}^{(p,k)})$ as the approximation error between $\mathcal{F}(p,k,s)$ and $\mathcal{F}(p,k)$. Theorem 1 establishes the convergence rate of $e(\hat{\boldsymbol{f}},\boldsymbol{f}^{(p,k)})$ as $n,p,k\rightarrow\infty$.

Theorem 1**.**

Assume $r=\{\log(pk)/n\}^{1/2}\rightarrow 0$ as $n,p,k\rightarrow\infty$. For linear learning with the $L_{1}$ penalty, $e_{\ell}(\hat{\boldsymbol{f}},\boldsymbol{f}^{(p,k)})=O[\max\{skr\log(r^{-1}),d_{n,p,k}\}]$, almost surely under $\mathbb{P}$. For the $L_{2}$ penalty, $e_{\ell}(\hat{\boldsymbol{f}},\boldsymbol{f}^{(p,k)})=O[\max\{(ps)^{1/2}kr\log(r^{-1}),d_{n,p,k}\}]$, almost surely under $\mathbb{P}$.

In Theorem 1, $s$ controls the balance between the estimation error, that is $skr\log(r^{-1})$ or $(ps)^{1/2}kr\log(r^{-1})$, and the approximation error $d_{n,p,k}$. As $s$ increases, $d_{n,p,k}$ decreases. The best trade off is one such that $skr\log(r^{-1})\sim d_{n,p,k}$ for the $L_{1}$ penalty, and $(ps)^{1/2}kr\log(r^{-1})\sim d_{n,p,k}$ for the $L_{2}$ penalty. The convergence of the excess $\ell$-risk requires that $k=o(n^{1/2})$ and $\log(p)=o(n)$ for the $L_{1}$ penalized method, and $k=o(n^{1/2})$ and $p=o(n)$ for the $L_{2}$ method.

Theorem 1 suggests that classification with a large number of classes can be very difficult. This helps to shed some light on the usefulness of our refine option. In particular, if a set of class labels frequently appears in set predictions (for instance, see Examples 2 and 3 in Section 6), one can consider a refined classification problem (with labels restricted in the prediction set) and use a richer functional space $\mathcal{F}$ if desired. Theorem 1 suggests that the new classifier can have better performance since the number of classes is smaller.

When $k$ is bounded, and the classification signal is sparse, Theorem 1 demonstrates the effectiveness of the $L_{1}$ method: it can be verified that if the true classification signal is sparse, then one can choose a large enough but fixed $s$, such that the approximation error is [math]. In other words, $\boldsymbol{f}^{(p,k)}\in\mathcal{F}(p,k,s)$ for some $s<\infty$. In this case, Theorem 1 can be greatly simplified.

Corollary 1**.**

Assume that $k$ is bounded, and the true classification signal depends on finitely many predictors. Assume $r^{\prime}=\{\log(p)/n\}^{1/2}\rightarrow 0$ as $n,p\rightarrow\infty$. We can choose $s=s^{*}$ for all large $n$, such that $d_{n,p,k}=0$. Consequently, for the $L_{1}$ penalty, $e_{\ell}(\hat{\boldsymbol{f}},\boldsymbol{f}^{(p,k)})=O\{r^{\prime}\log(r^{\prime-1})\}$, almost surely under $\mathbb{P}$, and for the $L_{2}$ penalty, $e_{\ell}(\hat{\boldsymbol{f}},\boldsymbol{f}^{(p,k)})=O\{p^{1/2}r^{\prime}\log(r^{\prime-1})\}$, almost surely under $\mathbb{P}$.

On the other hand, for $k\rightarrow\infty$ as $n\rightarrow\infty$, we cannot have a fixed $s$ such that $d_{n,p,k}=0$, even if the dimensionality $p$ is bounded. The next corollary considers a special situation where the number of true signal grows linearly with the number of classes. In this case, we can let $s=O(k)$, such that the approximation error is zero.

Corollary 2**.**

Consider any classification sub-problem where the label $y$ is restricted in $\{1,\ldots,k_{0}\}$, for any $1<k_{0}<k$. Suppose that the classification signal for the restricted sub-problem depends on at most $ck_{0}$ predictors, where $c$ is a fixed positive integer that is universal for all $k_{0}$. Then for the complete problem with $k\rightarrow\infty$ classes, one can choose $s=Ck$ with a fixed constant $C>0$, such that the approximation error $d_{n,p,k}=0$. Consequently, $e_{\ell}(\hat{\boldsymbol{f}},\boldsymbol{f}^{(p,k)})=O\{k^{2}r\log(r^{-1})\}$ for the $L_{1}$ penalty, and $O\{p^{1/2}k^{3/2}r\log(r^{-1})\}$ for the $L_{2}$ penalty, almost surely under $\mathbb{P}$.

A common scenario in which the assumptions of Corollary 2 hold is when each class has its own identifying attributes, and the number of signature attributes for each class is uniformly bounded by $c$. For instance, in cancer research, one may identify each cancer subtype with mutations on a small and non-overlapping group of feature genes. In this case, we can choose $s$ as a linear function of $k$ such that the approximation error is [math]. Another insight of Corollary 2 is that when there is no noise variable, that is, when $p=O(k)$, we have that the performance of the $L_{2}$ and $L_{1}$ regularization methods is comparable since the corresponding estimation errors have the same convergence rate.

Next, we study the convergence rate of the excess $\ell$-risk for kernel learning. To this end, we impose an assumption that the kernel is separable, and its corresponding kernel function is uniformly upper bounded. In other words, $K(\cdot,\cdot)<\infty$. Steinwart and Scovel (2007) and Blanchard et al. (2008), among others, used a similar assumption.

For kernel learning with the squared norm penalty, recall from Section 4.1 that the estimated classification functions are of the form $f_{q}(\boldsymbol{x})=\sum_{i=1}^{n}\theta_{q,i}K(\boldsymbol{x}_{i},\boldsymbol{x})+\theta_{q,0}$ with $q=1,\ldots,k-1$. We define $\mathcal{F}(p,k,s)=\{\boldsymbol{f}=(f_{1},\ldots,f_{k-1})^{T}:f_{q}=\sum_{i=1}^{n}\theta_{q,i}K(\boldsymbol{x}_{i},\boldsymbol{x})+\theta_{q,0},\ J(\boldsymbol{f})\leq s\}$, where $J(\boldsymbol{f})=\sum_{q=1}^{k-1}\boldsymbol{\theta}_{q}^{T}\boldsymbol{K}\boldsymbol{\theta}_{q}+\sum_{q=1}^{k-1}\theta_{q,0}^{2}$. Note that the intercepts are included in the penalty. In the RKHS learning literature, many theoretical results are derived without the intercept term (Bousquet and Elisseeff, 2002; Chen et al., 2004; Steinwart and Christmann, 2008). Our theory can incorporate regularized intercepts in the classification functions, hence is more general. Let $\mathcal{F}(p,k)$, $\boldsymbol{f}^{(p,k)}$ and $e(\boldsymbol{f},\boldsymbol{f}^{(p,k)})$ be defined analogously as in the linear learning case. The next theorem gives the convergence rate of the excess $\ell$-risk for kernel learning.

Theorem 2**.**

Assume $r=\{\log(k)/n\}^{1/2}\rightarrow 0$ as $n,k\rightarrow\infty$. For RKHS learning, assume that the kernel is separable, and the corresponding kernel function is uniformly upper bounded. We then have $e_{\ell}(\hat{\boldsymbol{f}},\boldsymbol{f}^{(p,k)})=O[\max\{skr\log(r^{-1}),d_{n,p,k}\}]$, almost surely under $\mathbb{P}$.

In Theorem 2, the dimension of the predictors $p$ does not directly affect the estimation error $skr\log(r^{-1})$. Instead, it is implicitly involved in the approximation error $d_{n,p,k}$. This is because the proof of Theorem 2 relies on the complexity of the function space $\mathcal{F}(p,k,s)$, in terms of its covering number (van der Vaart and Wellner, 2000). Note that Theorem 2 requires only that the kernel is separable and the kernel function is upper bounded, hence can be very general. On the other hand, if we restrict our consideration on a specific kernel, then more refined results can be obtained. For instance, many theoretical properties of the well known Gaussian kernel have been established. In Zhou (2002) and Steinwart and Scovel (2007), the relation between the covering number of the corresponding function space and $p$ has been obtained. Therefore, one can modify the proof of Theorem 2 and explore the explicit effect of $p$ on the estimation error accordingly.

So far, we have obtained the convergence rate of the estimation error for our classifiers. For linear learning and kernel learning, the rate can be close to the parametric rate $O(n^{-1/2})$, if $p$ and $k$ are negligible as $n\rightarrow\infty$. In the next section, we consider stronger assumptions, including a low noise assumption for multicategory classification problems. We show that faster rates are possible under these additional conditions.

5.2 Fast Rate under Low Noise Assumption

In the literature, many theoretical results have been established for binary SVMs with assumptions similar to Tsybakov’s margin condition (see Steinwart and Scovel, 2007; Bartlett and Wegkamp, 2008; Wegkamp and Yuan, 2011; Zhao et al., 2012, and the references therein.) In this paper, we consider the margin condition in multicategory problems with a reject option for a general loss function $\ell_{1}$ in (3). We show that when the classification function is in certain RKHSs, for example the Gaussian kernel space, a faster rate of convergence of the excess $\ell$-risk can be obtained.

Assumption 1**.**

(Low noise assumption) For $k$-class classification problems, we say that the distribution $\mathbb{P}(\boldsymbol{x},y)$ satisfies the margin condition at threshold level $a$ with exponent $\alpha>0$, if there exists a constant $c\geq 1$ such that for all $t>0$,

[TABLE]

Intuitively, under Assumption 1 with large $\alpha$, little probability mass is put around the boundary between the reject region and its complement. Thus, the classification signal is strong, and we expect that the estimation error can have a faster convergence rate. For binary SVM with a reject option, (8) reduces to the low noise assumption introduced in Bartlett and Wegkamp (2008) with $a=(1-d)/d$.

Because we intend to consider a general loss function, we impose some minor restrictions on the loss and some assumptions on the distribution. The next assumption is needed to prevent $|\langle\mathcal{Y}_{j},\boldsymbol{f}^{*}\rangle|$ from being too large, which yields a lower bound for the second order derivative of $\ell_{1}$ at the theoretical minimizer $\boldsymbol{f}^{*}$.

Assumption 2**.**

The loss function $\ell_{1}(u)$ in (3) is twice differentiable for $u<0$. Furthermore, for any $\boldsymbol{x}\in[0,1]^{\infty}$, the class conditional probability for any class $j$ is bounded away from [math]. In other words, $P_{j}(\boldsymbol{x})\geq\eta_{0}$ for a small and positive $\eta_{0}$.

Theorem 3 improves the convergence rate under the new assumptions.

Theorem 3**.**

Suppose Assumptions 1 and 2 hold with exponent $\alpha$. Then for our proposed method (4) with the Gaussian kernel, we have $e_{\ell}(\hat{\boldsymbol{f}},\boldsymbol{f}^{*})=O[\max\{skn^{-(1+\alpha)/(2+\alpha)},d_{n,p,k}\}]$.

Hence, the estimation error can converge at a rate faster than $n^{-1/2}$. In particular, for a problem with fixed $k$ and $\boldsymbol{f}^{(p,k)}\in\mathcal{F}(p,k,s)$ for a non-diverging $s$, if $\alpha\rightarrow\infty$, then the rate can become arbitrarily close to $n^{-1}$.

We remark that for a differentiable loss function $\ell_{1}$ whose derivative is strictly positive for small $u$, one may have $\langle\boldsymbol{f}^{*},\mathcal{Y}_{(1)}\rangle\rightarrow\infty$ if some $P_{j}(\boldsymbol{x})$ goes to zero. In this perspective, Assumption 2 helps to bound $\langle\boldsymbol{f}^{*},\mathcal{Y}_{(1)}\rangle$. However, Assumption 2 may not be needed for some special loss functions. For example, if $\ell_{1}$ is the reversed hinge loss, or the reversed FLAME loss proposed by Qiao and Zhang (2015), we can drop Assumption 2 while the result in Theorem 3 remains valid. In general, if the loss function $\ell_{1}$ is flat for small enough $u$, we can remove Assumption 2 from Theorem 3. See the proof and discussion of Theorem 3 in the Supplementary Materials for more discussions.

6 Numerical Studies

In this section, we study the numerical performance of our proposed classifiers (one with a reject option only, and one with both reject and refinement options.) For classification problems with weak signals, we show that the empirical [math]-$d$-$1$ loss for classifiers with a reject option can be smaller than that for regular classifiers. Furthermore, we show that the refine option can often provide refined set prediction with very high accuracy. Due to its reliable performance, in practical problems, the refinement option can be used to identify classes that are highly confusable with each other, so that future tests can be dedicated to these classes for potential improvement in classification accuracy.

6.1 Method of Comparisons

For all numerical problems in the current section, we study the performance of regular classifiers, classifiers with only the reject option, and classifiers with both reject and refine options. There are three possible prediction outcomes: (definite) label predictions, (refined) set predictions and rejections. Different types of predictions are shown using different colors in Figure 4. It can be seen that different classifiers have different capacity: regular classifiers can only provide label predictions while our classifiers with a refinement option can yield all three prediction types. We report classification performance on three disjoint subsets of observations, namely, $p1$, $p2$ and $p3$. The three subsets are defined as the observations which are label predicted, set predicted and rejected, respectively, by the classifier with both reject and refine options.

We report the misclassification error for each observation subset for each classifier. No misclassification rate is reported for rejected observations. For observations that are refined by our classifier ($p2$), we report the mis-refinement rate, which is defined as the proportion of observations whose true class labels are not in the prediction sets. We also report the empirical [math]-$d$-$1$ loss for the whole test data set for each classifier, where we count cost $1$ for each misclassification or mis-refinement, and cost $d$ for each rejection. The proportions of $p_{1}$, $p_{2}$ and $p_{3}$ are reported, since one may want to avoid large proportions of $p_{2}$ and $p_{3}$ unless necessary. We also report the proportion and mis-refinement rate for selected sets of class labels when they are of interest for the discussion.

For comparison purpose, we also use classifiers with probability estimation, and plug in the estimates into the Bayes rule (in Proposition 2) to achieve a reject option. The proportion of the label predicted and rejected observations by this approach are calculated, and the misclassification rate for the label prediction set is reported. The overall empirical [math]-$d$-$1$ loss is reported as well.

We conduct 100 replications for each example and report the average.

6.2 Simulations

We consider three simulated examples to assess the performance of the proposed methods. We focus on linear learning here and consider the Soft-LUM classifier loss (Soft; Liu et al., 2011), the DWD loss, and the SVM loss. Each loss is associated with one regular classifier, one with rejection only and one with both reject and refine options. Moreover, we implement the probability estimation method associated with Theorem 3 of Zhang and Liu (2014).

To select the best tuning parameters $s$ and $\delta$, we choose from a candidate set $\Lambda\times\Delta$ the best pair that minimizes the empirical [math]-$d$-$1$ loss on a separate tuning data set, where $\Lambda$ consists of $30$ $\lambda$ values, and $\Delta=\{0.3,0.25,\ldots,0.05,0\}\cdot\max_{i,j}|\langle\mathcal{Y}_{j},\hat{\boldsymbol{f}}(\boldsymbol{x}_{i})\rangle|$. The multiplicative constant $\max_{i,j}|\langle\mathcal{Y}_{j},\hat{\boldsymbol{f}}(\boldsymbol{x}_{i})\rangle|$ is used to scale for the magnitude of the angle margins. This is because when $p$ is large, a severe regularization $J(\boldsymbol{f})$ is often needed, which would shrink the magnitude of $\hat{\boldsymbol{f}}$ (Zhang et al., 2013). In this case, using a fixed set of candidate values in $\Delta$ could be suboptimal. Note that letting $\delta=0$ shuts off reject and refine options. To illustrate the effect of $a$ on the reject and refinement results, we fit the classifiers with several $a$ values between $a_{1}$ and $a_{2}$, but show the results for the best one only to save some space. More details are included in the Supplementary Materials.

Example 1: A four-class example with equal prior probabilities. We first generate two covariates that determine the true class distributions. In particular, $\boldsymbol{x}\mid Y=j$, $j=1,2,3,4$, are uniformly distributed in $[-0.3,1]\times[-0.3,1]$, $[-0.3,1]\times[-1,0.3]$, $[-1,0.3]\times[-1,0.3]$, and $[-1,0.3]\times[-0.3,1]$ respectively. See the left panel of Figure 5 for a typical example on the first two dimensions. We then add 98 noise covariates. The training and tuning data sets are of size $150$ respectively, and the test data set is of size $12000$. In this example we let $d=0.6$, and report the behavior of the Soft loss using the $L_{1}$ penalty only.

Example 2: A three-class example with equal prior probabilities. The true classification signal depends on two predictors, and the marginal distributions of $\boldsymbol{X}\mid Y=j$, $j=1,2,3$, follow \frac{1}{3}N\big{(}(-\sqrt{3}/2,1/2)^{T},\sigma^{2}I_{2}\big{)}+\frac{1}{3}N\big{(}(-1,0)^{T},\sigma^{2}I_{2}\big{)}+\frac{1}{3}N\big{(}(0,0)^{T},\sigma^{2}I_{2}\big{)}, \frac{1}{3}N\big{(}(-\sqrt{3}/2,-1/2)^{T},\sigma^{2}I_{2}\big{)}+\frac{1}{3}N\big{(}(-1,0)^{T},\sigma^{2}I_{2}\big{)}+\frac{1}{3}N\big{(}(0,0)^{T},\sigma^{2}I_{2}\big{)}, and \frac{2}{3}N\big{(}(1,0)^{T},\sigma^{2}I_{2}\big{)}+\frac{1}{3}N\big{(}(0,0)^{T},\sigma^{2}I_{2}\big{)} respectively where $\sigma=0.2$. We then add 398 noise covariates. The training and tuning sample sizes are both $120$, and the test sample size is $12000$. See the middle panel of Figure 5. In this example we let $d=0.5$, and report the results for the SVM (hinge) loss using the $L_{2}$ penalty only. Many observations are confusable only between classes 1 and 2, hence the proportion and error rate for predictions $\{1,2\}$ are reported separately.

Example 3: A four-class problem where the classification signal between classes 1 and 2, and that between classes 3 and 4, are confusable. We let the class label depend on only two predictors. In particular, for class $j$, the marginal distribution of $x_{1}$ and $x_{2}$ is normal with $\sigma=0.2$, and the corresponding mean is uniformly distributed on the line segment between $(0,0)^{T}$ and $(z_{1}^{j},z_{2}^{j})^{T};\ j=1,\ldots,4$, where $(z_{1}^{1},z_{2}^{1})=(1,0.2)$, $(z_{1}^{2},z_{2}^{2})=(1,-0.2)$, $(z_{1}^{3},z_{2}^{3})=(-1,0.2)$, and $(z_{1}^{4},z_{2}^{4})=(-1,-0.2)$. See the right panel of Figure 5. We then add 98 noise covariates. The training and tuning data sets are both of size $160$, and the test data set size is $10000$. In this example, we choose $d=0.5$, and report the performance of DWD loss with the $L_{1}$ penalty. The proportions and error rates of prediction $\{1,2\}$ and $\{3,4\}$ are reported.

All the noise covariates added are $i.i.d.$ $N(0,0.01)$. To save space, we only report selected results here in Table 1, while more results can be found in the Supplementary Materials. We collect some key observations below.

$\bullet$  For the probability estimation plug-in method, its overall error is greater than our proposed methods, and can be even greater than that of the regular classifier without a reject option. This is because for high-dimensional problems, accurate probability estimation is too difficult, leading to degenerated performance for classification.

$\bullet$  Our proposed classifiers with the rejection option can lead to less overall error, compared to traditional classifiers which only provide label predictions. The large values of the errors on $p_{3}$ for the regular classifiers (all of which are greater than 50%), indicates that the reject option is able to identify the set of testing observations which are most difficult to be classified.

$\bullet$  The refine option can provide very accurate set prediction, with very low mis-refinement rates. The usefulness of the classifiers with both the reject and refine options is also reflected by the decreased overall error rate.

Compared to the classifier with only the reject option, the classifier with both reject and refine options can avoid misclassification for subset $p_{2}$: this can be seen from the reduction from misclassification rate 45.35% (51.00%, 35.71%, resp.) to mis-refinement rate 1.581% (11.26%, 1.771%, resp.) 2. 2.

Another advantage of the refine option is that it can identify class labels that are most confusing to each other. In Example 2, 73.1% of the size 2 set predictions are $\{1,2\}$, while class 1 and class 2 indeed exist a two-way confusion. In Example 3, about 40% of the size 2 set predictions are $\{1,2\}$ (and another 40% for $\{3,4\}$.) Hence, a researcher can conclude that the intrinsic difference between classes $1$ and $2$, or $3$ and $4$, is relatively small. In genetic research, this information can be used to verify that two diseases are similar, or can be used to introduce new studies on the corresponding causations.

$\bullet$  As the parameter $a$ grows (not shown here for the sake of space), our proposed methods become more conservative. In particular, the proportion of rejected observations increases as $a$ increases. On the other hand, the effect of $a$ on the classification performance changes under various settings, and there is no single $a$ that works uniformly the best for all problems. Our numerical experience shows that the best $a$ for a given problem is often at either $a_{1}$ or $a_{2}$ (defined in Proposition 3.) Hence, for real applications, we recommend to train the classifier with $a=a_{1}$ and $a=a_{2}$, and select the one with the better performance.

6.3 Real Data Analysis

In this section, we illustrate the use of our methods for the Glioblastoma Multiforme Cancer data set (GBM, Verhaak et al., 2010) and the normalized handwritten digits data set scanned from envelopes by the U.S. Postal Service (ZIP, Hastie et al., 2009).

In the GBM data set, there are 4 subtypes of Glioblastoma Multiforme cancer, namely, Classical, Mesenchymal, Neural and Proneural, and within each type we have 92, 111, 56, 97 patients, respectively. The gene expression levels on 16548 genes are measured as predictors to characterize the cancer subtypes. We normalize the data set so that each predictor has mean 0 and sample variance 1. As a demonstration, we use $d=0.4$, the Soft loss, and the $L_{2}$ penalty. To select the best tuning parameters, we split the data set into 6 groups of observations whose sizes are roughly the same, choose one group as the test data set, and perform 5-fold cross validations on the remaining observations. We report the average result over 100 random splits. To alleviate the computational burden, we choose 2000 genes with the greatest median absolute deviation values based on the training sample for each split.

We include selected results for the GBM data set on the top half of Table 2. More results can be found in the Supplementary Materials. Our proposed reject and refine options can often abstain from making label predictions on observations on which the classification signal is weak, which leads to a reduced overall loss. The mis-refinement rate on the refined observations ($p_{2}$) is very small compared to the label prediction error rates by the regular classifier and the reject only classifier. More interestingly, most of the set predictions occur for either $\{\textrm{Classical},\textrm{Mesenchymal}\}$ or $\{\textrm{Neural},\textrm{Proneural}\}$. This suggests that the GBM subtypes Classical and Mesenchymal, or Neural and Proneural, share some common characteristics in their genotypes. This finding is consistent with both The Cancer Genome Atlas core samples or validation samples in Verhaak et al. (2010).

The ZIP data set has been extensively studied by many previous works. We choose categories “3”, “4” and “9” to demonstrate the effect of the refine option. For handwritten digits, it is sometimes difficult for machines to classify between “4” and “9”, while the difference between “3” and “4” or “3” and “9” is more obvious. For visualization, we draw a PCA plot for the test data on the left panel of Figure 6. In the middle panel, we provide a scatter plot by projecting the sample to the 2D space using $\hat{\boldsymbol{f}}(\boldsymbol{x})\in\mathbb{R}^{2}$. In particular, observations with reject or refined set predications are shown in red squares. It can be seen that the observations which are refined are precisely those sitting on 2-way classification boundaries (shown as the dashed red lines), while most of them are between “4” and “9”. In the analysis, we use $d=0.4$, the DWD loss, and the $L_{2}$ penalty. We normalize the data set before the analysis. To select the best tuning parameters, we split the training data set into two groups, and use one to train the classifier and the other for tuning. We report the average results of 100 splits.

The results for the ZIP data set are reported in the second half of Table 2, while more comprehensive results can be found in the Supplementary Materials. Note that although there are only a few rejected observations ($<0.368\%$ on average), their misclassification rate is as high as 95.14%, if not rejected. This stunningly high error rate justifies our reject option. Though there are only 2.578% observations that are refined, the mis-refinement rate is as low as 0.110%, almost always correct. The middle panel of Figure 2 also suggests that the refinement decision is well deserved since the refined data points are in close vicinity to the classification boundaries. Lastly, it can be seen that, for quite a few observations, the classification signal is very vague between “4” and “9”, which is consistent with our common sense (see the middle and right panels of Figure 6).

7 Conclusion

In this article, we enrich regular multicategory classification methods with reject and refine options. While a reject option adds to the capacity of an existing multicategory classifier, a refine option has the potential to open a new direction. Usually, statistical learning researches have been aiming to create an “ultimate” machine with perfect predictive power. However, sometimes the nature of the data or the data collection process has determined that a significant misclassification is inevitable for some observations. Many methods have been proposed but the obtained improvement is somewhat limited. When the cost of misclassification is too large to bear, it may be wise to take a step back and start to think of new ideas out of the box. A refine option may have opened a door to these. With a refine option, one can often successfully identify observations with a subset of most confusable class labels. Future resources can be allocated to these observations with a refined set of labels to acquire new data with better quality.

The aim of this article is to establish a new framework for classification. Many different loss functions and penalty functions may be incorporated into this framework for the purpose of reject and refine options. We provide a novel statistical learning theory, with emphasis on diverging dimensions and diverging numbers of classes. Future work will be devoted to how to better utilize the refined set predictions. Many new research topics will follow this new learning tool.

Appendix

In this appendix, we provide brief outlines of the proofs of Theorems 1, 2 and 3.

**Theorem 1 and Theorem 2

**There are two major steps in the proof of Theorems 1 and 2. The first step is to decompose the excess $\ell$-risk into the estimation error and approximation error. Then we show that the probability of the estimation error exceeding $O(skr\log(r^{-1}))$ for the $L_{1}$ penalty, or $O(\sqrt{ps}kr\log(r^{-1}))$ for the $L_{2}$ penalty, can be written in terms of a concentration inequality indexed by a scaled empirical process. The second step is to obtain a suitable probability upper bound of this concentration inequality. To this end, one can use the chaining technique, which discretizes the functional space of the optimization problem, hence decomposing the corresponding probability into several parts. For each part, the probability can be controlled by established concentration inequalities. See Theorem A.2 in Wang and Shen (2007) for an example.

Therefore, the question boils down to control the complexity of the discretized functional space. A common approach to depict such complexity in the literature is to use the entropy numbers. In the Supplementary Materials, for linear and kernel learning, we introduce Lemmas 2 and 4 respectively, to control the complexity of the corresponding functional spaces for the empirical processes, in terms of their $L_{2}$ entropy numbers. In particular, we show that for a small and positive $\epsilon$, the $\epsilon$-entropy numbers for linear and kernel learning are in the order of $O(\epsilon^{-2})$ under mild conditions. Consequently, we can prove the desired concentration inequality.

It should be noted that, although the orders of the entropy numbers for linear and kernel learning are similar, the techniques used are quite different. In particular, in linear learning, we treat the functional space as a convex hull of $2p$ functions, which leads to a bound on the entropy number. For kernel learning, we consider the natural embedding of the kernel function into the regular $L_{2}$ functional space consisting of continuous functions on the domain of $\boldsymbol{x}$. Such embedding can be shown to be absolutely 2-summing with 2-summing norm no larger than 1. Hence we can bound the entropy number of this embedding operator (which can be shown to be the same as the entropy number of the original kernel space) by its corresponding approximation numbers, which can be further bounded by Carl’s inequality between approximation and entropy numbers.

**Theorem 3

**Theorem 3 extends the well established results on fast rate of convergence from binary classifiers to multicategory ones. The key to the proof is to find a pseudo-norm that can be used to both upper and lower bound the conditional excess $\ell$-risk $g_{\boldsymbol{f}}(\boldsymbol{x},y)=\sum_{j\neq y}\ell\{\langle\boldsymbol{f},\mathcal{Y}_{j}\rangle\}-\sum_{j\neq y}\ell\{\langle\boldsymbol{f}^{*},\mathcal{Y}_{j}\rangle\}$ (up to constants). In Bartlett and Wegkamp (2008), as the modified hinge loss function $\psi(u)$ is piecewise linear, and remains flat for large $u$, one can use $\rho(f_{1},f_{2})\propto|f_{1}-f_{2}|$ as the pseudo-norm. However, for more general loss functions, especially differentiable loss functions, an $L_{1}$ type pseudo-norm cannot lower bound the conditional excess $\ell$-risk. Therefore, we employ the (squared) $L_{2}$ type pseudo-norm in this proof. With the low noise assumption, we can show that the class $\{g_{\boldsymbol{f}}(\boldsymbol{x},y)\}$ is a Bernstein class with the Bernstein exponent $\alpha/(1+\alpha)$. The next step is to apply the symmetrization technique, and show that the estimation error can be (up to a constant) bounded by a tail probability plus a small term that converges to zero at a very fast speed, where the tail probability term is indexed by an empirical process of $\{g_{\boldsymbol{f}}(\boldsymbol{x},y)\}$. At this stage, we can employ Bernstein’s inequality to bound the corresponding tail probability. As $\{g_{\boldsymbol{f}}(\boldsymbol{x},y)\}$ is a Bernstein class, the variance term in the power of the upper bound in Bernstein’s inequality can be bounded by a linear term of $\mathbb{E}g_{\boldsymbol{f}}(\boldsymbol{x},y)$. Combined with an upper bound on the entropy number for Gaussian kernel space, we can prove the desired result in Theorem 3.

Supplementary Materials

SM1:

Detailed proofs of Proposition 1, Proposition 3, Theorem 1, Theorem 2 and Theorem 3; derivations of the implementations; extended numerical results. (.pdf file)