Multi-Task Representation Learning for Conservative Linear Bandits

TL;DR

This paper introduces a novel framework and algorithm for safe multi-task linear bandits that leverage shared low-dimensional representations, with theoretical guarantees and experimental validation.

Contribution

It proposes the CMTRL framework and Safe-AltGDmin algorithm for conservative linear bandits, incorporating safety constraints into multi-task representation learning.

Findings

The algorithm effectively recovers low-rank feature matrices under safety constraints.

Theoretical regret and sample complexity bounds are established for the proposed framework.

Experimental results demonstrate improved performance over benchmark algorithms.

Abstract

This paper presents the Constrained Multi-Task Representation Learning (CMTRL) framework for linear bandits. We consider T linear bandit tasks in a d dimensional space, which share a common low-dimensional representation of dimension r, where r is much smaller than the minimum of d and T. Furthermore, tasks are constrained so that only actions meeting specific safety or performance requirements are allowed, referred to as conservative (safe) bandits. We introduce a novel algorithm, Safe-Alternating projected Gradient Descent and minimization (Safe-AltGDmin), to recover a low-rank feature matrix while satisfying the given constraints. Building on this algorithm, we propose a multi-task representation learning framework for conservative linear bandits and establish theoretical guarantees for its regret and sample complexity bounds. We presented experiments and compared the performance of our algorithm with benchmark algorithms.