RLHF: A comprehensive Survey for Cultural, Multimodal and Low Latency Alignment Methods

TL;DR

This survey reviews recent advances in Reinforcement Learning from Human Feedback (RLHF), emphasizing multi-modal, cultural fairness, and low-latency alignment methods to improve the robustness and equity of large language models.

Contribution

It provides a comprehensive synthesis of new RLHF techniques beyond text, analyzing foundational algorithms and outlining open challenges in the field.

Findings

Comparison of PPO, DPO, and GRPO algorithms

Identification of gaps in multi-modal and cultural alignment

Discussion of low-latency optimization challenges

Abstract

Reinforcement Learning from Human Feedback (RLHF) is the standard for aligning Large Language Models (LLMs), yet recent progress has moved beyond canonical text-based methods. This survey synthesizes the new frontier of alignment research by addressing critical gaps in multi-modal alignment, cultural fairness, and low-latency optimization. To systematically explore these domains, we first review foundational algo- rithms, including PPO, DPO, and GRPO, before presenting a detailed analysis of the latest innovations. By providing a comparative synthesis of these techniques and outlining open challenges, this work serves as an essential roadmap for researchers building more robust, efficient, and equitable AI systems.