Reinforcement Learning

TL;DR

Reinforcement learning (RL) is a versatile framework for adaptive decision-making, with methods like value-based and policy search approaches, and extensions for risk-averse scenarios and unknown rewards, applicable in diverse domains.

Contribution

This chapter provides a comprehensive overview of RL fundamentals, main approaches, and recent extensions, highlighting the diversity and adaptability of RL methods.

Findings

RL effectively applied in games and autonomous vehicles

Two main RL approaches: value-based and policy search

Extensions include risk-averse RL and scenarios with unknown rewards

Abstract

Reinforcement learning (RL) is a general framework for adaptive control, which has proven to be efficient in many domains, e.g., board games, video games or autonomous vehicles. In such problems, an agent faces a sequential decision-making problem where, at every time step, it observes its state, performs an action, receives a reward and moves to a new state. An RL agent learns by trial and error a good policy (or controller) based on observations and numeric reward feedback on the previously performed action. In this chapter, we present the basic framework of RL and recall the two main families of approaches that have been developed to learn a good policy. The first one, which is value-based, consists in estimating the value of an optimal policy, value from which a policy can be recovered, while the other, called policy search, directly works in a policy space. Actor-critic methods can be seen as a policy search technique where the policy value that is learned guides the policy improvement. Besides, we give an overview of some extensions of the standard RL framework, notably when risk-averse behavior needs to be taken into account or when rewards are not available or not known.