FAST: Similarity-based Knowledge Transfer for Efficient Policy Learning

TL;DR

FAST is a transfer learning framework that uses visual and textual task representations to estimate similarity, enabling efficient policy transfer and reducing training time in evolving environments like racing games.

Contribution

The paper introduces FAST, a novel similarity-based transfer learning method that leverages visual and textual data for efficient policy transfer in dynamic domains.

Findings

FAST achieves competitive performance with fewer training steps.

Embedding-driven similarity estimation improves transfer effectiveness.

Method reduces computational costs in evolving environments.

Abstract

Transfer Learning (TL) offers the potential to accelerate learning by transferring knowledge across tasks. However, it faces critical challenges such as negative transfer, domain adaptation and inefficiency in selecting solid source policies. These issues often represent critical problems in evolving domains, i.e. game development, where scenarios transform and agents must adapt. The continuous release of new agents is costly and inefficient. In this work we challenge the key issues in TL to improve knowledge transfer, agents performance across tasks and reduce computational costs. The proposed methodology, called FAST - Framework for Adaptive Similarity-based Transfer, leverages visual frames and textual descriptions to create a latent representation of tasks dynamics, that is exploited to estimate similarity between environments. The similarity scores guides our method in choosing candidate policies from which transfer abilities to simplify learning of novel tasks. Experimental results, over multiple racing tracks, demonstrate that FAST achieves competitive final performance compared to learning-from-scratch methods while requiring significantly less training steps. These findings highlight the potential of embedding-driven task similarity estimations.