Proper-Composite Loss Functions in Arbitrary Dimensions

TL;DR

This paper extends the analysis of proper-composite loss functions from finite to infinite-dimensional spaces, providing a unified framework for understanding their properties in complex machine learning tasks.

Contribution

It generalizes existing finite-dimensional results to infinite-dimensional settings, offering a new characterization of the canonical link in density and conditional density estimation.

Findings

Extended proper-composite loss analysis to infinite dimensions

Unified framework for loss functions in complex models

Simplified characterization of the canonical link

Abstract

The study of a machine learning problem is in many ways is difficult to separate from the study of the loss function being used. One avenue of inquiry has been to look at these loss functions in terms of their properties as scoring rules via the proper-composite representation, in which predictions are mapped to probability distributions which are then scored via a scoring rule. However, recent research so far has primarily been concerned with analysing the (typically) finite-dimensional conditional risk problem on the output space, leaving aside the larger total risk minimisation. We generalise a number of these results to an infinite dimensional setting and in doing so we are able to exploit the familial resemblance of density and conditional density estimation to provide a simple characterisation of the canonical link.