Online Transfer Learning in Reinforcement Learning Domains

TL;DR

This paper introduces an online transfer learning framework for reinforcement learning, unifying existing methods and providing theoretical convergence guarantees, with empirical validation of the proposed approach.

Contribution

It presents a novel online transfer framework that generalizes existing transfer methods in reinforcement learning and offers convergence proofs for various algorithms.

Findings

Convergence of Q-learning and Sarsa with tabular representation proven.

Convergence of Q-learning and Sarsa with linear function approximation established.

Teaching does not harm asymptotic performance.

Abstract

This paper proposes an online transfer framework to capture the interaction among agents and shows that current transfer learning in reinforcement learning is a special case of online transfer. Furthermore, this paper re-characterizes existing agents-teaching-agents methods as online transfer and analyze one such teaching method in three ways. First, the convergence of Q-learning and Sarsa with tabular representation with a finite budget is proven. Second, the convergence of Q-learning and Sarsa with linear function approximation is established. Third, the we show the asymptotic performance cannot be hurt through teaching. Additionally, all theoretical results are empirically validated.