HLER: Human-in-the-Loop Economic Research via Multi-Agent Pipelines for Empirical Discovery

TL;DR

HLER presents a multi-agent system that combines automation with human oversight to facilitate scalable and feasible empirical research in economics, emphasizing dataset-aware hypothesis generation and iterative review processes.

Contribution

The paper introduces HLER, a novel multi-agent architecture that integrates human oversight with automated empirical research workflows, emphasizing dataset-aware hypothesis generation and iterative refinement.

Findings

Dataset-aware hypothesis generation achieves 87% feasibility.

Complete manuscripts produced at $0.8-$1.5 per run.

Human-in-the-loop design improves research feasibility.

Abstract

Large language models (LLMs) have enabled agent-based systems that aim to automate scientific research workflows. Most existing approaches focus on fully autonomous discovery, where AI systems generate research ideas, conduct analyses, and produce manuscripts with minimal human involvement. However, empirical research in economics and the social sciences poses additional constraints: research questions must be grounded in available datasets, identification strategies require careful design, and human judgment remains essential for evaluating economic significance. We introduce HLER (Human-in-the-Loop Economic Research), a multi-agent architecture that supports empirical research automation while preserving critical human oversight. The system orchestrates specialized agents for data auditing, data profiling, hypothesis generation, econometric analysis, manuscript drafting, and automated review. A key design principle is dataset-aware hypothesis generation, where candidate research questions are constrained by dataset structure, variable availability, and distributional diagnostics, reducing infeasible or hallucinated hypotheses. HLER further implements a two-loop architecture: a question quality loop that screens and selects feasible hypotheses, and a research revision loop where automated review triggers re-analysis and manuscript revision. Human decision gates are embedded at key stages, allowing researchers to guide the automated pipeline. Experiments on three empirical datasets show that dataset-aware hypothesis generation produces feasible research questions in 87% of cases (versus 41% under unconstrained generation), while complete empirical manuscripts can be produced at an average API cost of $0.8-$1.5 per run. These results suggest that Human-AI collaborative pipelines may provide a practical path toward scalable empirical research.