An Introduction to Deep Reinforcement and Imitation Learning

TL;DR

This paper introduces the core concepts, algorithms, and techniques of Deep Reinforcement Learning and Deep Imitation Learning, focusing on their application to embodied agents like robots and virtual characters.

Contribution

It provides an in-depth, self-contained overview of foundational algorithms in DRL and DIL, emphasizing understanding over broad survey coverage.

Findings

Explains key algorithms like REINFORCE and PPO for DRL.

Describes imitation learning methods such as Behavioral Cloning and GAIL.

Provides mathematical foundations for decision-making in embodied agents.

Abstract

Embodied agents, such as robots and virtual characters, must continuously select actions to execute tasks effectively, solving complex sequential decision-making problems. Given the difficulty of designing such controllers manually, learning-based approaches have emerged as promising alternatives, most notably Deep Reinforcement Learning (DRL) and Deep Imitation Learning (DIL). DRL leverages reward signals to optimize behavior, while DIL uses expert demonstrations to guide learning. This document introduces DRL and DIL in the context of embodied agents, adopting a concise, depth-first approach to the literature. It is self-contained, presenting all necessary mathematical and machine learning concepts as they are needed. It is not intended as a survey of the field; rather, it focuses on a small set of foundational algorithms and techniques, prioritizing in-depth understanding over broad coverage. The material ranges from Markov Decision Processes to REINFORCE and Proximal Policy Optimization (PPO) for DRL, and from Behavioral Cloning to Dataset Aggregation (DAgger) and Generative Adversarial Imitation Learning (GAIL) for DIL.