Policy-Controlled Generalized Share: A General Framework with a Transformer Instantiation for Strictly Online Switching-Oracle Tracking

TL;DR

This paper introduces PCGS, a flexible online learning framework using a Transformer for adaptive control, achieving low regret in non-stationary environments and outperforming baselines in synthetic and real-world tests.

Contribution

It proposes a novel framework with a Transformer-based instantiation for online prediction, providing theoretical regret guarantees and empirical superior performance.

Findings

PCGS-TF achieves the lowest mean dynamic regret across seven non-stationary families.

PCGS-TF attains the lowest normalized dynamic regret on household electricity data for multiple switch counts.

The framework provides pathwise weighted regret guarantees with adaptive post-loss controls.

Abstract

Static regret to a single expert is often the wrong target for strictly online prediction under non-stationarity, where the best expert may switch repeatedly over time. We study Policy-Controlled Generalized Share (PCGS), a general strictly online framework in which the generalized-share recursion is fixed while the post-loss update controls are allowed to vary adaptively. Its principal instantiation in this paper is PCGS-TF, which uses a causal Transformer as an update controller: after round t finishes and the loss vector is observed, the Transformer outputs the controls that map w_t to w_{t+1} without altering the already committed decision w_t. Under admissible post-loss update controls, we obtain a pathwise weighted regret guarantee for general time-varying learning rates, and a standard dynamic-regret guarantee against any expert path with at most S switches under the constant-learning-rate specialization. Empirically, on a controlled synthetic suite with exact dynamic-programming switching-oracle evaluation, PCGS-TF attains the lowest mean dynamic regret in all seven non-stationary families, with its advantage increasing for larger expert pools. On a reproduced household-electricity benchmark, PCGS-TF also achieves the lowest normalized dynamic regret for S = 5, 10, and 20.