An Axiomatic Approach to Loss Aggregation and an Adapted Aggregating Algorithm

TL;DR

This paper introduces a new axiomatic framework for loss aggregation in online learning, proposing a tailored Aggregating Algorithm that handles generalized quasi-sum aggregation functions with strong theoretical guarantees.

Contribution

It characterizes a broad class of loss aggregation functions as quasi-sums and develops a modified Aggregating Algorithm suitable for these functions, extending theoretical properties.

Findings

The proposed algorithm recovers Bayes' updating.

It provides a time-independent regret bound.

Generalized aggregations reflect the learner's attitude towards losses.

Abstract

Supervised learning has gone beyond the expected risk minimization framework. Central to most of these developments is the introduction of more general aggregation functions for losses incurred by the learner. In this paper, we turn towards online learning under expert advice. Via easily justified assumptions we characterize a set of reasonable loss aggregation functions as quasi-sums. Based upon this insight, we suggest a variant of the Aggregating Algorithm tailored to these more general aggregation functions. This variant inherits most of the nice theoretical properties of the AA, such as recovery of Bayes' updating and a time-independent bound on quasi-sum regret. Finally, we argue that generalized aggregations express the attitude of the learner towards losses.