MIR: Methodology Inspiration Retrieval for Scientific Research Problems

TL;DR

This paper introduces Methodology Inspiration Retrieval (MIR), a novel approach to find prior research that can inspire solutions for scientific problems, leveraging a new dataset, the Methodology Adjacency Graph, and LLM-based re-ranking to improve retrieval effectiveness.

Contribution

It presents a new MIR task, constructs a dedicated dataset, develops the Methodology Adjacency Graph for capturing methodological lineage, and enhances retrieval with graph-based embeddings and LLM re-ranking.

Findings

Significant improvements in retrieval metrics (+5.4 Recall@3, +7.8 mAP) using MAG-based embeddings.

Additional gains (+4.5 Recall@3, +4.8 mAP) from LLM-based re-ranking.

Demonstrates MIR's potential to advance automated scientific discovery.

Abstract

There has been a surge of interest in harnessing the reasoning capabilities of Large Language Models (LLMs) to accelerate scientific discovery. While existing approaches rely on grounding the discovery process within the relevant literature, effectiveness varies significantly with the quality and nature of the retrieved literature. We address the challenge of retrieving prior work whose concepts can inspire solutions for a given research problem, a task we define as Methodology Inspiration Retrieval (MIR). We construct a novel dataset tailored for training and evaluating retrievers on MIR, and establish baselines. To address MIR, we build the Methodology Adjacency Graph (MAG); capturing methodological lineage through citation relationships. We leverage MAG to embed an "intuitive prior" into dense retrievers for identifying patterns of methodological inspiration beyond superficial semantic similarity. This achieves significant gains of +5.4 in Recall@3 and +7.8 in Mean Average Precision (mAP) over strong baselines. Further, we adapt LLM-based re-ranking strategies to MIR, yielding additional improvements of +4.5 in Recall@3 and +4.8 in mAP. Through extensive ablation studies and qualitative analyses, we exhibit the promise of MIR in enhancing automated scientific discovery and outline avenues for advancing inspiration-driven retrieval.