A Method for Fast Autonomy Transfer in Reinforcement Learning

TL;DR

This paper presents MCAC, a reinforcement learning method that enables rapid transfer of autonomy by leveraging pre-trained critic functions, significantly reducing adaptation time and improving reward outcomes.

Contribution

Introduction of the MCAC algorithm that uses multiple critic value functions for fast RL adaptation without extensive retraining.

Findings

MCAC achieves up to 22.76x faster transfer.

MCAC outperforms baseline algorithms in reward accumulation.

Empirical results validate the effectiveness of MCAC.

Abstract

This paper introduces a novel reinforcement learning (RL) strategy designed to facilitate rapid autonomy transfer by utilizing pre-trained critic value functions from multiple environments. Unlike traditional methods that require extensive retraining or fine-tuning, our approach integrates existing knowledge, enabling an RL agent to adapt swiftly to new settings without requiring extensive computational resources. Our contributions include development of the Multi-Critic Actor-Critic (MCAC) algorithm, establishing its convergence, and empirical evidence demonstrating its efficacy. Our experimental results show that MCAC significantly outperforms the baseline actor-critic algorithm, achieving up to 22.76x faster autonomy transfer and higher reward accumulation. This advancement underscores the potential of leveraging accumulated knowledge for efficient adaptation in RL applications.