LMAR: Language Model Augmented Retriever for Domain-specific Knowledge Indexing

TL;DR

LMAR introduces a versatile framework combining LLM-guided data synthesis and contrastive embedding adaptation to improve domain-specific knowledge retrieval efficiently and effectively.

Contribution

It presents a novel, model-agnostic pipeline that enhances retrieval performance in domain-specific contexts using synthetic data and clustering, with minimal hardware needs.

Findings

Outperforms baseline models on multiple domain datasets

Maintains low latency and moderate hardware requirements

Seamlessly integrates with existing RAG architectures

Abstract

Retrieval Augmented Generation (RAG) systems often struggle with domain-specific knowledge due to performance deterioration of pre-trained embeddings and prohibitive computational costs of large language model (LLM)-based retrievers. While fine-tuning data augmentation embedding models offers a promising direction, its effectiveness is limited by the need for high-quality training data and reliable chunking strategies that preserve contextual integrity. We propose LMAR (Language Model Augmented Retriever), a model-agnostic framework that addresses these challenges by combining LLM-guided data synthesis with contrastive embedding adaptation and efficient text clustering. LMAR consists of a two-stage pipeline: (1) Triplet sampling and synthetic data augmentation, where LLMs act as both labeler and validator to ensure high-fidelity supervision throughout the pipeline. Experimental results across multiple domain-specific benchmark datasets demonstrate that LMAR outperforms multiple baseline models, while maintaining moderate hardware requirements and low latency. Its model-agnostic nature further enables seamless integration with emerging RAG architectures and text embedding models, ensuring continual improvements without redesigning the pipeline. These results highlight LMAR as a practical and cost-effective solution for scalable domain-specific adaptation.