Learning Shared Representations for Multi-Task Linear Bandits

TL;DR

This paper presents a novel multi-task linear bandit algorithm that learns shared low-rank representations to improve sample efficiency and reduce regret across multiple related tasks.

Contribution

It introduces a new OFUL-based algorithm leveraging shared low-rank structures with theoretical guarantees and improved regret bounds.

Findings

Achieves (\,drNT) regret bound, better than independent task solutions.

Provides spectral initialization for shared model estimation.

Validates performance through numerical simulations.

Abstract

Multi-task representation learning is an approach that learns shared latent representations across related tasks, facilitating knowledge transfer and improving sample efficiency. This paper introduces a novel approach to multi-task representation learning in linear bandits. We consider a setting with T concurrent linear bandit tasks, each with feature dimension d, that share a common latent representation of dimension r \ll min{d,T}$, capturing their underlying relatedness. We propose a new Optimism in the Face of Uncertainty Linear (OFUL) algorithm that leverages shared low-rank representations to enhance decision-making in a sample-efficient manner. Our algorithm first collects data through an exploration phase, estimates the shared model via spectral initialization, and then conducts OFUL based learning over a newly constructed confidence set. We provide theoretical guarantees for the confidence set and prove that the unknown reward vectors lie within the confidence set with high probability. We derive cumulative regret bounds and show that the proposed approach achieves \tilde{O}(\sqrt{drNT}), a significant improvement over solving the T tasks independently, resulting in a regret of \tilde{O}(dT\sqrt{N}). We performed numerical simulations to validate the performance of our algorithm for different problem sizes.