Information-Consistent Language Model Recommendations through Group Relative Policy Optimization

TL;DR

This paper introduces a reinforcement learning method using Group Relative Policy Optimization to enhance the consistency of large language models in delivering invariant information across semantically equivalent prompts, crucial for enterprise applications.

Contribution

It adapts GRPO to enforce information stability in LLMs, introducing entropy-based rewards and prompt grouping, a novel approach for improving consistency in enterprise contexts.

Findings

GRPO-fine-tuned model shows reduced variability compared to baseline.

The approach effectively enforces information stability across prompt groups.

First application of GRPO for aligning LLMs towards information consistency.

Abstract

Large Language Models (LLMs) are increasingly deployed in business-critical domains such as finance, education, healthcare, and customer support, where users expect consistent and reliable recommendations. Yet LLMs often exhibit variability when prompts are phrased with minor differences, even when semantically equivalent. Such inconsistency undermines trust, complicates compliance, and disrupts user experience. While personalization is desirable in certain contexts, many enterprise scenarios, such as HR onboarding, customer support, or policy disclosure, require invariant information delivery regardless of phrasing or prior conversational history. Existing approaches, including retrieval-augmented generation (RAG) and temperature tuning, improve factuality or reduce stochasticity, but cannot guarantee stability across equivalent prompts. In this paper, we propose a reinforcement learning framework based on Group Relative Policy Optimization (GRPO) to directly optimize for consistency. Unlike prior applications of GRPO, which have been limited to reasoning and code generation, we adapt GRPO to enforce the stability of information content across groups of semantically equivalent prompts. We introduce entropy-based helpfulness and stability rewards, treating prompt variants as groups and resetting conversational context to isolate phrasing effects. Experiments on investment and job recommendation tasks show that our GRPO-fine-tuned model reduces variability compared to the baseline LLM model. To our knowledge, this is a novel application of GRPO for aligning LLMs toward information consistency, reframing variability not as an acceptable feature of generative diversity, but as a correctable flaw in enterprise deployments.