Misspecified Universal Learning

TL;DR

This paper investigates universal learning under model misspecification with log-loss, analyzing minimax regret and proposing an optimal learner within a unified framework for various uncertainty settings.

Contribution

It extends existing theories to the misspecified setting, deriving minimax regret and designing an optimal universal learner applicable across multiple learning modes.

Findings

Derived the minimax regret in the misspecified setting.

Proposed an optimal universal learner for broad uncertainty scenarios.

Unified framework applicable to online, batch, supervised, and unsupervised learning.

Abstract

This paper addresses the problem of universal learning under model misspecification with log-loss. In this setting, the learner operates with a hypothesis class of models denoted by $\Theta$, while the true data-generating process belongs to a broader class $\Phi \supset \Theta$, and may lie outside the assumed hypothesis space. Classical approaches have characterized the minimax regret and identified optimal universal learners in both the well-specified stochastic and individual deterministic frameworks. The misspecified setting has received comparatively less attention, although several important results have emerged in recent years. Extending these foundations, we analyze the minimax regret in the misspecified setting and derive the corresponding optimal universal learner. We propose this formulation as a unified framework for universal learning, applicable to any form of uncertainty in the data-generating process, across both online and batch data arrival modes, as well as supervised and unsupervised learning tasks.