Fast learning rates for plug-in classifiers under the margin condition

TL;DR

This paper demonstrates that plug-in classifiers can achieve super-fast convergence rates under the margin condition, challenging previous conjectures about their limitations and establishing optimal bounds.

Contribution

It proves that plug-in classifiers can attain rates faster than $n^{-1}$ under the margin condition, contradicting prior beliefs and conjectures.

Findings

Constructed plug-in classifiers with super-fast rates

Established minimax lower bounds for convergence rates

Challenged previous conjectures about classifier convergence

Abstract

It has been recently shown that, under the margin (or low noise) assumption, there exist classifiers attaining fast rates of convergence of the excess Bayes risk, i.e., the rates faster than $n^{-1/2}$. The works on this subject suggested the following two conjectures: (i) the best achievable fast rate is of the order $n^{-1}$, and (ii) the plug-in classifiers generally converge slower than the classifiers based on empirical risk minimization. We show that both conjectures are not correct. In particular, we construct plug-in classifiers that can achieve not only the fast, but also the {\it super-fast} rates, i.e., the rates faster than $n^{-1}$. We establish minimax lower bounds showing that the obtained rates cannot be improved.