Pistis-RAG: Enhancing Retrieval-Augmented Generation with Human Feedback

TL;DR

Pistis-RAG is a novel retrieval-augmented generation framework that leverages structured human feedback to improve content relevance and generation quality, addressing limitations of traditional RAG systems.

Contribution

It introduces a content-centric RAG framework that effectively utilizes human feedback for better alignment with user preferences and improved performance.

Findings

Achieves a 6.06% increase in MMLU accuracy

Achieves a 7.08% increase in C-EVAL accuracy

Outperforms baseline RAG in content alignment

Abstract

RAG systems face limitations when semantic relevance alone does not guarantee improved generation quality. This issue becomes particularly evident due to the sensitivity of large language models (LLMs) to the ordering of few-shot prompts, which can affect model performance. To address this challenge, aligning LLM outputs with human preferences using structured feedback, such as options to copy, regenerate, or dislike, offers a promising method for improvement. This feedback is applied to the entire list of inputs rather than giving specific ratings for individual documents, making it a Listwide Labels Learning-to-Rank task. To address this task, we propose Pistis-RAG, a new RAG framework designed with a content-centric approach to better align LLMs with human preferences. Pistis-RAG effectively utilizes human feedback, enhancing content ranking and generation quality. To validate our framework, we use public datasets to simulate human feedback, allowing us to evaluate and refine our method effectively. Experimental results indicate that Pistis-RAG improves alignment with human preferences relative to the baseline RAG system, showing a 6.06% increase in MMLU (English) and a 7.08% increase in C-EVAL (Chinese) accuracy metrics. These results highlight Pistis-RAG's effectiveness in overcoming the limitations associated with traditional RAG approaches.