Revisiting the Solution of Meta KDD Cup 2024: CRAG

TL;DR

This paper details the solution developed by team APEX for the Meta KDD CUP 2024 CRAG challenge, introducing a routing-based adaptive RAG pipeline that enhances performance on diverse, dynamic QA tasks.

Contribution

It proposes a novel routing-based domain and dynamic adaptive RAG pipeline to better handle diverse and dynamic questions in retrieval-augmented generation systems.

Findings

Achieved 2nd place in Task 2&3 on the leaderboard.

Developed a comprehensive RAG evaluation benchmark.

Demonstrated superior performance with the proposed method.

Abstract

This paper presents the solution of our team APEX in the Meta KDD CUP 2024: CRAG Comprehensive RAG Benchmark Challenge. The CRAG benchmark addresses the limitations of existing QA benchmarks in evaluating the diverse and dynamic challenges faced by Retrieval-Augmented Generation (RAG) systems. It provides a more comprehensive assessment of RAG performance and contributes to advancing research in this field. We propose a routing-based domain and dynamic adaptive RAG pipeline, which performs specific processing for the diverse and dynamic nature of the question in all three stages: retrieval, augmentation, and generation. Our method achieved superior performance on CRAG and ranked 2nd for Task 2&3 on the final competition leaderboard. Our implementation is available at this link: https://github.com/USTCAGI/CRAG-in-KDD-Cup2024.