Logic-Oriented Retriever Enhancement via Contrastive Learning

TL;DR

LORE employs contrastive learning to enhance retriever models by emphasizing logical structure in embeddings, significantly improving knowledge retrieval for complex queries without external supervision.

Contribution

It introduces a novel, self-supervised contrastive learning method that activates logical reasoning capacity in retrievers, improving their performance on knowledge-intensive tasks.

Findings

Improves retrieval accuracy on complex logical queries

Enhances downstream generation quality

Requires no external supervision or resources

Abstract

Large language models (LLMs) struggle in knowledge-intensive tasks, as retrievers often overfit to surface similarity and fail on queries involving complex logical relations. The capacity for logical analysis is inherent in model representations but remains underutilized in standard training. LORE (Logic ORiented Retriever Enhancement) introduces fine-grained contrastive learning to activate this latent capacity, guiding embeddings toward evidence aligned with logical structure rather than shallow similarity. LORE requires no external upervision, resources, or pre-retrieval analysis, remains index-compatible, and consistently improves retrieval utility and downstream generation while maintaining efficiency. The datasets and code are publicly available at https://github.com/mazehart/Lore-RAG.