Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QA

TL;DR

This paper introduces a reinforced co-adaptation framework for industrial advertising QA that jointly optimizes retrieval and generation, significantly reducing hallucinations and improving accuracy, safety, and user engagement in real-world deployment.

Contribution

It proposes a novel Graph-aware Retrieval and evidence-constrained reinforcement learning approach tailored for industrial knowledge, enhancing faithfulness and safety in advertising QA systems.

Findings

72% reduction in hallucination rate

28.6% increase in like rate in online tests

System deployed in production for over six months

Abstract

Industrial advertising question answering (QA) is a high-stakes task in which hallucinated content, particularly fabricated URLs, can lead to financial loss, compliance violations, and legal risk. Although Retrieval-Augmented Generation (RAG) is widely adopted, deploying it in production remains challenging because industrial knowledge is inherently relational, frequently updated, and insufficiently aligned with generation objectives. We propose a reinforced co-adaptation framework that jointly optimizes retrieval and generation through two components: (1) Graph-aware Retrieval (GraphRAG), which models entity-relation structure over a high-citation knowledge subgraph for multi-hop, domain-specific evidence selection; and (2) evidence-constrained reinforcement learning via Group Relative Policy Optimization (GRPO) with multi-dimensional rewards covering faithfulness, style compliance, safety, and URL validity. Experiments on an internal advertising QA dataset show consistent gains across expert-judged dimensions including accuracy, completeness, and safety, while reducing the hallucination rate by 72\%. A two-week online A/B test demonstrates a 28.6\% increase in like rate, a 46.2\% decrease in dislike rate, and a 92.7\% reduction in URL hallucination. The system has been running in production for over half a year and has served millions of QA interactions.