A Survey On Enhancing Reinforcement Learning in Complex Environments: Insights from Human and LLM Feedback

TL;DR

This survey reviews how human and large language model feedback can improve reinforcement learning in complex, high-dimensional environments, addressing challenges like sample inefficiency and slow learning.

Contribution

It provides a comprehensive overview of methods integrating human and LLM feedback into RL to enhance performance in complex environments with large observation spaces.

Findings

Feedback from humans and LLMs improves RL decision-making.

Integrating feedback accelerates learning and enhances resilience.

Addresses challenges of high-dimensional observation spaces.

Abstract

Reinforcement learning (RL) is one of the active fields in machine learning, demonstrating remarkable potential in tackling real-world challenges. Despite its promising prospects, this methodology has encountered with issues and challenges, hindering it from achieving the best performance. In particular, these approaches lack decent performance when navigating environments and solving tasks with large observation space, often resulting in sample-inefficiency and prolonged learning times. This issue, commonly referred to as the curse of dimensionality, complicates decision-making for RL agents, necessitating a careful balance between attention and decision-making. RL agents, when augmented with human or large language models' (LLMs) feedback, may exhibit resilience and adaptability, leading to enhanced performance and accelerated learning. Such feedback, conveyed through various modalities or granularities including natural language, serves as a guide for RL agents, aiding them in discerning relevant environmental cues and optimizing decision-making processes. In this survey paper, we mainly focus on problems of two-folds: firstly, we focus on humans or an LLMs assistance, investigating the ways in which these entities may collaborate with the RL agent in order to foster optimal behavior and expedite learning; secondly, we delve into the research papers dedicated to addressing the intricacies of environments characterized by large observation space.