Sequential Transfer in Multi-armed Bandit with Finite Set of Models

TL;DR

This paper introduces a new algorithm for sequential transfer in multi-armed bandit problems, enabling online learning agents to leverage prior task knowledge to minimize cumulative regret across tasks.

Contribution

It presents a novel bandit algorithm using a method-of-moments approach for task estimation, advancing transfer learning in online multi-task settings.

Findings

Derived regret bounds for the proposed algorithm

Demonstrated improved performance over baseline methods

Established theoretical guarantees for transfer effectiveness

Abstract

Learning from prior tasks and transferring that experience to improve future performance is critical for building lifelong learning agents. Although results in supervised and reinforcement learning show that transfer may significantly improve the learning performance, most of the literature on transfer is focused on batch learning tasks. In this paper we study the problem of \textit{sequential transfer in online learning}, notably in the multi-armed bandit framework, where the objective is to minimize the cumulative regret over a sequence of tasks by incrementally transferring knowledge from prior tasks. We introduce a novel bandit algorithm based on a method-of-moments approach for the estimation of the possible tasks and derive regret bounds for it.