Learning to Solve Tasks with Exploring Prior Behaviours

TL;DR

This paper introduces IRDEC, a method enabling agents to learn necessary prior behaviors through exploration, improving task-solving in sparse reward scenarios without extra demonstrations of those behaviors.

Contribution

The paper presents IRDEC, a novel approach that allows agents to acquire prior behaviors via intrinsic rewards, enhancing their ability to solve tasks with varied initial conditions.

Findings

IRDEC outperforms baselines on navigation tasks.

IRDEC effectively learns prior behaviors without additional demonstrations.

The method improves success rates in robotic manipulation with sparse rewards.

Abstract

Demonstrations are widely used in Deep Reinforcement Learning (DRL) for facilitating solving tasks with sparse rewards. However, the tasks in real-world scenarios can often have varied initial conditions from the demonstration, which would require additional prior behaviours. For example, consider we are given the demonstration for the task of \emph{picking up an object from an open drawer}, but the drawer is closed in the training. Without acquiring the prior behaviours of opening the drawer, the robot is unlikely to solve the task. To address this, in this paper we propose an Intrinsic Rewards Driven Example-based Control \textbf{(IRDEC)}. Our method can endow agents with the ability to explore and acquire the required prior behaviours and then connect to the task-specific behaviours in the demonstration to solve sparse-reward tasks without requiring additional demonstration of the prior behaviours. The performance of our method outperforms other baselines on three navigation tasks and one robotic manipulation task with sparse rewards. Codes are available at https://github.com/Ricky-Zhu/IRDEC.