BSharedRAG: Backbone Shared Retrieval-Augmented Generation for the E-commerce Domain

TL;DR

BSharedRAG introduces a shared backbone framework for retrieval-augmented generation in e-commerce, enhancing performance by jointly optimizing retrieval and generation tasks with a domain-specific backbone.

Contribution

The paper proposes a novel shared backbone RAG framework with domain-specific pre-training and plug-and-play LoRA modules for improved retrieval and generation in e-commerce.

Findings

Outperforms baselines by 5-13% in retrieval metrics

Achieves 23% improvement in BLEU-3 for generation

Demonstrates effectiveness on two e-commerce datasets

Abstract

Retrieval Augmented Generation (RAG) system is important in domains such as e-commerce, which has many long-tail entities and frequently updated information. Most existing works adopt separate modules for retrieval and generation, which may be suboptimal since the retrieval task and the generation task cannot benefit from each other to improve performance. We propose a novel Backbone Shared RAG framework (BSharedRAG). It first uses a domain-specific corpus to continually pre-train a base model as a domain-specific backbone model and then trains two plug-and-play Low-Rank Adaptation (LoRA) modules based on the shared backbone to minimize retrieval and generation losses respectively. Experimental results indicate that our proposed BSharedRAG outperforms baseline models by 5% and 13% in Hit@3 upon two datasets in retrieval evaluation and by 23% in terms of BLEU-3 in generation evaluation. Our codes, models, and dataset are available at https://bsharedrag.github.io.