A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions

TL;DR

This survey reviews reinforcement learning techniques for large language models in data-scarce scenarios, proposing a hierarchical framework and taxonomy to guide future research in data-efficient RL methods.

Contribution

It provides the first systematic review and a hierarchical framework for understanding data-efficient reinforcement learning for LLMs under data scarcity.

Findings

Developed a taxonomy of existing RL methods for LLMs.

Analyzed strengths and limitations of different approaches.

Outlined future research directions for scalable RL in LLMs.

Abstract

Reinforcement learning (RL) has emerged as a powerful post-training paradigm for enhancing the reasoning capabilities of large language models (LLMs). However, reinforcement learning for LLMs faces substantial data scarcity challenges, including the limited availability of high-quality external supervision and the constrained volume of model-generated experience. These limitations make data-efficient reinforcement learning a critical research direction. In this survey, we present the first systematic review of reinforcement learning for LLMs under data scarcity. We propose a bottom-up hierarchical framework built around three complementary perspectives: the data-centric perspective, the training-centric perspective, and the framework-centric perspective. We develop a taxonomy of existing methods, summarize representative approaches in each category, and analyze their strengths and limitations. Our taxonomy aims to provide a clear conceptual foundation for understanding the design space of data-efficient RL for LLMs and to guide researchers working in this emerging area. We hope this survey offers a comprehensive roadmap for future research and inspires new directions toward more efficient and scalable reinforcement learning post-training for LLMs.