Linear Probability Forecasting

TL;DR

This paper introduces two efficient online algorithms for multi-class classification using linear and kernelized models, with theoretical guarantees and experimental comparisons to logistic regression.

Contribution

It presents novel online algorithms for multi-class classification with Brier loss, including a kernelized version with proven theoretical guarantees.

Findings

Algorithms outperform logistic regression in experiments

Theoretical bounds on loss are established for both algorithms

Kernelized algorithm extends the applicability to non-linear models

Abstract

Multi-class classification is one of the most important tasks in machine learning. In this paper we consider two online multi-class classification problems: classification by a linear model and by a kernelized model. The quality of predictions is measured by the Brier loss function. We suggest two computationally efficient algorithms to work with these problems and prove theoretical guarantees on their losses. We kernelize one of the algorithms and prove theoretical guarantees on its loss. We perform experiments and compare our algorithms with logistic regression.