DS@GT at TREC TOT 2025: Bridging Vague Recollection with Fusion Retrieval and Learned Reranking

TL;DR

This paper presents a two-stage retrieval system combining hybrid retrieval, topical partitioning, and learned reranking, including LLM-based methods, to improve performance on the TREC ToT task.

Contribution

It introduces a novel fusion retrieval approach with topic-aware indexing and LLM-based reranking for the TREC ToT challenge.

Findings

Achieved recall of 0.66 and NDCG@1000 of 0.41 on test set.

Demonstrated the effectiveness of combining hybrid retrieval with learned reranking.

Generated 5000 synthetic queries to train rerankers.

Abstract

We develop a two-stage retrieval system that combines multiple complementary retrieval methods with a learned reranker and LLM-based reranking, to address the TREC Tip-of-the-Tongue (ToT) task. In the first stage, we employ hybrid retrieval that merges LLM-based retrieval, sparse (BM25), and dense (BGE-M3) retrieval methods. We also introduce topic-aware multi-index dense retrieval that partitions the Wikipedia corpus into 24 topical domains. In the second stage, we evaluate both a trained LambdaMART reranker and LLM-based reranking. To support model training, we generate 5000 synthetic ToT queries using LLMs. Our best system achieves recall of 0.66 and NDCG@1000 of 0.41 on the test set by combining hybrid retrieval with Gemini-2.5-flash reranking, demonstrating the effectiveness of fusion retrieval.