SHRAG: AFrameworkfor Combining Human-Inspired Search with RAG

TL;DR

SHRAG is a novel framework that enhances Retrieval-Augmented Generation by automating query transformation and multilingual retrieval, leading to more accurate and reliable multi-lingual question answering.

Contribution

It introduces SHRAG, which combines human-inspired search strategies with RAG, automating query reformulation and multilingual retrieval to improve performance.

Findings

Significantly improves RAG accuracy and reliability.

Enables efficient cross-lingual question answering.

Demonstrates potential for a new direct response search paradigm.

Abstract

Retrieval-Augmented Generation (RAG) is gaining recognition as one of the key technological axes for next generation information retrieval, owing to its ability to mitigate the hallucination phenomenon in Large Language Models (LLMs)and effectively incorporate up-to-date information. However, specialized expertise is necessary to construct ahigh-quality retrieval system independently; moreover, RAGdemonstratesrelativelyslowerprocessing speeds compared to conventional pure retrieval systems because it involves both retrieval and generation stages. Accordingly, this study proposes SHRAG, a novel framework designed to facilitate the seamless integration of Information Retrieval and RAG while simultaneously securing precise retrieval performance. SHRAG utilizes a Large Language Model as a Query Strategist to automatically transform unstructured natural language queries into logically structured search queries, subsequently performing Boolean retrieval to emulate the search process of an expert human searcher. Furthermore, it incorporates multilingual query expansion and a multilingual embedding model, enabling it to perform efficient cross-lingual question answering within the multilingual dataset environment of the ScienceON Challenge. Experimental results demonstrate that the proposed method, combining logical retrieval capabilities and generative reasoning, can significantly enhance the accuracy and reliability of RAG systems. Furthermore, SHRAG movesbeyondconventionaldocument-centric retrieval methods, presenting the potential for a new search paradigm capable of providing direct and reliable responses to queries.