Learning From Non-iid Data: Fast Rates for the One-vs-All Multiclass Plug-in Classifiers

TL;DR

This paper establishes fast learning rates for multiclass plug-in classifiers trained on non-iid data, such as strongly mixing or drifting distributions, under a multiclass margin assumption, extending previous binary-class results.

Contribution

It introduces new theoretical learning rates for multiclass plug-in classifiers in non-iid settings, independent of the number of classes, generalizing prior binary-class findings.

Findings

Fast rates achieved under strongly mixing data

Results hold for drifting distributions

Rates are independent of class number

Abstract

We prove new fast learning rates for the one-vs-all multiclass plug-in classifiers trained either from exponentially strongly mixing data or from data generated by a converging drifting distribution. These are two typical scenarios where training data are not iid. The learning rates are obtained under a multiclass version of Tsybakov's margin assumption, a type of low-noise assumption, and do not depend on the number of classes. Our results are general and include a previous result for binary-class plug-in classifiers with iid data as a special case. In contrast to previous works for least squares SVMs under the binary-class setting, our results retain the optimal learning rate in the iid case.