Transfer Learning for Customized Car Racing Environments

TL;DR

This paper investigates transfer learning in deep reinforcement learning for car racing, demonstrating improved performance and faster convergence across different circuits, with a comparison of model-based and model-free methods.

Contribution

It introduces transfer learning techniques for deep reinforcement learning in car racing environments and compares model-based and model-free approaches.

Findings

Transfer learning boosts performance on target circuits.

Model-based approaches outperform model-free in speed and accuracy.

Transfer learning enables high performance during the learning process.

Abstract

Transfer Learning, a technique where a model/agent can use the knowledge/expertise that it gained from one task and exploit that to solve another closely-related task, is often used in tackling problems in deep learning. Through this project, we explore transfer learning in the purview of deep reinforcement learning. Specifically, we want to use transfer learning to achieve the fast lap times in OpenAI's Car racing environment by training the agent on one circuit, and racing it on other customized target environments by zero-shot transfer or by additional fine-tuning. In addition, we compare the performance of model-based and model-free approaches, and observe that model-based approaches dominate in performance and converge faster than model-free approaches in this environment. We observe that transfer learning in most setups not only boosts the performance on the target domain, but also shows high performance ability during learning.