Lecture Notes on Partially Known MDPs

TL;DR

This paper discusses methods for finding optimal policies in Markov decision processes when the model is only partially known, aiming to bridge offline and online reinforcement learning approaches.

Contribution

It introduces a framework for transitioning from offline to online learning in partially known MDPs, advancing reinforcement learning techniques.

Findings

Proposes a gradual transition strategy from offline to online learning.

Provides theoretical insights into learning in partially known MDPs.

Lays groundwork for practical reinforcement learning algorithms in uncertain environments.

Abstract

In these notes we will tackle the problem of finding optimal policies for Markov decision processes (MDPs) which are not fully known to us. Our intention is to slowly transition from an offline setting to an online (learning) setting. Namely, we are moving towards reinforcement learning.