A Survey on Offline Model-Based Reinforcement Learning

TL;DR

This survey reviews recent advances in offline model-based reinforcement learning, highlighting methods to address distributional shift and discussing future research directions in the field.

Contribution

It provides a comprehensive overview of recent offline model-based RL research, emphasizing approaches to mitigate distributional shift and identifying key challenges and future directions.

Findings

Summarizes recent key papers and their methods.

Highlights approaches to handle distributional shift.

Discusses future research challenges and directions.

Abstract

Model-based approaches are becoming increasingly popular in the field of offline reinforcement learning, with high potential in real-world applications due to the model's capability of thoroughly utilizing the large historical datasets available with supervised learning techniques. This paper presents a literature review of recent work in offline model-based reinforcement learning, a field that utilizes model-based approaches in offline reinforcement learning. The survey provides a brief overview of the concepts and recent developments in both offline reinforcement learning and model-based reinforcement learning, and discuss the intersection of the two fields. We then presents key relevant papers in the field of offline model-based reinforcement learning and discuss their methods, particularly their approaches in solving the issue of distributional shift, the main problem faced by all current offline model-based reinforcement learning methods. We further discuss key challenges faced by the field, and suggest possible directions for future work.