Transferring Domain Knowledge with an Adviser in Continuous Tasks

TL;DR

This paper introduces a method to incorporate domain knowledge into reinforcement learning agents via an adviser, accelerating learning and improving policy quality in continuous tasks.

Contribution

It adapts the DDPG algorithm to include an adviser, enabling explicit integration of pre-learned policies or relationships into the learning process.

Findings

Accelerates learning in benchmark tasks

Enhances policy quality towards better optima

Demonstrates effectiveness of adviser integration

Abstract

Recent advances in Reinforcement Learning (RL) have surpassed human-level performance in many simulated environments. However, existing reinforcement learning techniques are incapable of explicitly incorporating already known domain-specific knowledge into the learning process. Therefore, the agents have to explore and learn the domain knowledge independently through a trial and error approach, which consumes both time and resources to make valid responses. Hence, we adapt the Deep Deterministic Policy Gradient (DDPG) algorithm to incorporate an adviser, which allows integrating domain knowledge in the form of pre-learned policies or pre-defined relationships to enhance the agent's learning process. Our experiments on OpenAi Gym benchmark tasks show that integrating domain knowledge through advisers expedites the learning and improves the policy towards better optima.