Decision Making with Side Information and Unbounded Loss Functions

TL;DR

This paper develops a generalized decision-making framework incorporating side information and unbounded loss functions, providing conditions for consistent, exponentially convergent decisions applicable to estimation and system identification.

Contribution

It introduces a new model for decision-making with side information, extending PAC learning to unbounded loss functions and broadening application scope.

Findings

Conditions for exponential convergence in decision-making.

Bounded exponential Orlicz expectation ensures consistency.

Applications demonstrated in estimation and system identification.

Abstract

We consider the problem of decision-making with side information and unbounded loss functions. Inspired by probably approximately correct learning model, we use a slightly different model that incorporates the notion of side information in a more generic form to make it applicable to a broader class of applications including parameter estimation and system identification. We address sufficient conditions for consistent decision-making with exponential convergence behavior. In this regard, besides a certain condition on the growth function of the class of loss functions, it suffices that the class of loss functions be dominated by a measurable function whose exponential Orlicz expectation is uniformly bounded over the probabilistic model. Decay exponent, decay constant, and sample complexity are discussed. Example applications to method of moments, maximum likelihood estimation, and system identification are illustrated, as well.