Boosting Reinforcement Learning and Planning with Demonstrations: A Survey

TL;DR

This survey reviews how demonstrations can enhance reinforcement learning and planning by providing expert guidance, discussing methods of integration, collection strategies, and practical applications like the ManiSkill benchmark.

Contribution

It offers a comprehensive overview of demonstration-based methods in decision making, highlighting new approaches and practical pipelines for their generation and use.

Findings

Demonstrations improve learning efficiency in complex environments.

Various methods exist for integrating demonstrations into reinforcement learning and planning.

A practical pipeline for generating and utilizing demonstrations is demonstrated on the ManiSkill benchmark.

Abstract

Although reinforcement learning has seen tremendous success recently, this kind of trial-and-error learning can be impractical or inefficient in complex environments. The use of demonstrations, on the other hand, enables agents to benefit from expert knowledge rather than having to discover the best action to take through exploration. In this survey, we discuss the advantages of using demonstrations in sequential decision making, various ways to apply demonstrations in learning-based decision making paradigms (for example, reinforcement learning and planning in the learned models), and how to collect the demonstrations in various scenarios. Additionally, we exemplify a practical pipeline for generating and utilizing demonstrations in the recently proposed ManiSkill robot learning benchmark.