MARVEL: A Multi Agent-based Research Validator and Enabler using Large Language Models

TL;DR

MARVEL is an open-source, multi-agent framework that enhances scientific research and question answering by integrating retrieval, reasoning, and source tracking, specifically demonstrated in gravitational-wave research.

Contribution

It introduces a novel multi-agent system combining retrieval-augmented generation with Monte Carlo Tree Search for domain-aware scientific assistance.

Findings

Matches GPT-4o baseline on literature queries

Outperforms on detector-operations content

Provides publicly available datasets and framework

Abstract

We present MARVEL (https://ligogpt.mit.edu/marvel), a locally deployable, open-source framework for domain-aware question answering and assisted scientific research. It is designed to address the increasing demands of a digital assistant for scientific groups that can read highly technical data, cite precisely, and operate within authenticated networks. MARVEL combines a fast path for straightforward queries with a more deliberate DeepSearch mode that integrates retrieval-augmented generation and Monte Carlo Tree Search. It explores complementary subqueries, allocates more compute to promising branches, and maintains a global evidence ledger that preserves sources during drafting. We applied this framework in the context of gravitational-wave research related to the Laser Interferometer Gravitational-wave Observatory. Answers are grounded in a curated semantic index of research literature, doctoral theses, LIGO documents, and long-running detector electronic logbooks, with targeted web searches when appropriate. Because direct benchmarking against commercial LLMs cannot be performed on private data, we evaluated MARVEL on two publicly available surrogate datasets that capture comparable semantic and technical characteristics. On these benchmarks, MARVEL matches a GPT-4o mini baseline on literature-centric queries and substantially outperforms it on detector-operations content, where domain retrieval and guided reasoning are decisive. By making the complete framework and evaluation datasets openly available, we aim to provide a reproducible foundation for developing domain-specific scientific assistants.