Probing the Future of Meta-Analysis: Eliciting Design Principles via an Agentic Research IDE

TL;DR

This paper introduces Research IDE, an innovative authoring environment that integrates multi-agent verification and hypothesis breakpoints to enhance meta-analysis workflows, fostering researcher autonomy and leveraging AI for insight generation.

Contribution

It presents a novel research environment that combines multi-agent systems with hypothesis verification, emphasizing preserving researcher control and intellectual ownership.

Findings

Researchers preferred the verification workflow with breakpoints.

Breakpoints facilitated insights and active knowledge use.

Participants valued preserving autonomy in AI-assisted research.

Abstract

Meta-analyses and systematic reviews demand rigorous abductive reasoning to build, test, and refine hypotheses across vast, heterogeneous literature. While NLP advancements have automated parts of this pipeline, existing tools often detach researchers from the cognitive loop or function merely as retrieval engines, leading to loss of intellectual ownership and frequent context switching. We present Research IDE, a prototype reimagining authoring environments through the "Research as Code" metaphor. Research IDE embeds a multi-agent backend into the writing flow, enabling in-situ verification via "hypothesis breakpoints." A one-week field deployment with 8 domain experts, followed by a reflective workshop, as a Research through Design (RtD) probe, reveals that users strongly preferred this verification workflow, actively leveraged prior knowledge for confirmation, and reported that breakpoints sparked insights. Drawing from participant feedback and suggestions, we derive design implications for future AI-assisted research tools that fully preserve researcher autonomy and intellectual ownership while harnessing computational scale.