Non-Stationary Representation Learning in Sequential Linear Bandits

TL;DR

This paper introduces an online algorithm for non-stationary representation learning in sequential linear bandits, enabling efficient multi-task decision-making across changing environments with theoretical guarantees and empirical validation.

Contribution

It proposes a novel adaptive algorithm for learning and transferring non-stationary representations in sequential linear bandits, outperforming existing methods.

Findings

The algorithm achieves significant performance improvements over independent task approaches.

Theoretical analysis confirms the algorithm's efficiency in non-stationary settings.

Experimental results on synthetic and real data validate the approach's effectiveness.

Abstract

In this paper, we study representation learning for multi-task decision-making in non-stationary environments. We consider the framework of sequential linear bandits, where the agent performs a series of tasks drawn from distinct sets associated with different environments. The embeddings of tasks in each set share a low-dimensional feature extractor called representation, and representations are different across sets. We propose an online algorithm that facilitates efficient decision-making by learning and transferring non-stationary representations in an adaptive fashion. We prove that our algorithm significantly outperforms the existing ones that treat tasks independently. We also conduct experiments using both synthetic and real data to validate our theoretical insights and demonstrate the efficacy of our algorithm.