ReCoQA: A Benchmark for Tool-Augmented and Multi-Step Reasoning in Real Estate Question and Answering

TL;DR

ReCoQA is a comprehensive benchmark for evaluating multi-step, tool-augmented reasoning in real estate question answering, featuring structured supervision and a hierarchical agent framework.

Contribution

The paper introduces ReCoQA, a large-scale real estate QA benchmark with verifiable intermediate steps, and proposes HIRE-Agent, a hierarchical reasoning framework as a strong baseline.

Findings

HIRE-Agent effectively integrates heterogeneous evidence for complex reasoning.

Hierarchical collaboration improves performance on real-world reasoning tasks.

ReCoQA provides extensive supervision for multi-source information navigation.

Abstract

Developing agents capable of navigating fragmented, multi-source information remains challenging, primarily due to the scarcity of benchmarks reflecting hybrid workflows combining database querying with external APIs. To bridge this gap, we introduce ReCoQA, a large-scale benchmark of 29,270 real-estate instances featuring machine-verifiable supervision for intermediate steps, including structured intent labels, SQL queries, and API calls. Complementarily, we propose HIRE-Agent, a hierarchical framework instantiating an understand-plan-execute architecture as a strong baseline. By orchestrating a Front-end parser, a planning Supervisor, and execution Specialists, HIRE-Agent effectively integrates heterogeneous evidence. Extensive experiments demonstrate that HIRE-Agent constitutes a strong baseline and substantiates the necessity of hierarchical collaboration for complex, real-world reasoning tasks.