HiMA-Ecom: Enabling Joint Training of Hierarchical Multi-Agent E-commerce Assistants

TL;DR

This paper introduces HiMA-Ecom, a benchmark for hierarchical multi-agent e-commerce systems, and proposes HiMA-R1, a joint training method that improves efficiency and performance of LLM-based assistants.

Contribution

The paper presents the first hierarchical multi-agent e-commerce benchmark and a novel joint training method with variance reduction and adaptive memory mechanisms.

Findings

HiMA-R1 achieves comparable performance to larger models.

Our method surpasses previous models by an average of 6%.

Experiments validate the effectiveness of the proposed approach.

Abstract

Hierarchical multi-agent systems based on large language models (LLMs) have become a common paradigm for building AI assistants in vertical domains such as e-commerce, where a master agent coordinates multiple specialized sub-agents. Despite their practical importance, realistic benchmarks for training and evaluating such systems remain scarce, and joint optimization across functionally distinct agents is still challenging. To address this gap, we introduce HiMA-Ecom, the first hierarchical multi-agent benchmark tailored for e-commerce scenarios. HiMA-Ecom contains 22.8K instances, including agent-specific supervised fine-tuning samples with memory and system-level input-output pairs for joint multi-agent reinforcement learning. Building upon it, a joint training method named HiMA-R1 is proposed. It presents Variance-Reduction Group Relative Policy Optimization (VR-GRPO), which employs initial trajectory-based Monte Carlo sampling to mitigate the exponential joint action space and selects informative agent groups for efficient updates based on reward variance. Furthermore, an adaptive memory evolution mechanism that repurposes GRPO rewards as cost-free supervisory signals is designed to eliminate repetitive reasoning and accelerate convergence. Experiments on HiMA-Ecom demonstrate that our method, built upon smaller 3B/7B open-source models, achieves performance comparable to that of larger LLMs, such as DeepSeek-R1, and surpasses DeepSeek-V3 by an average of 6\%.