Towards Hierarchical Multi-Agent Workflows for Zero-Shot Prompt Optimization

TL;DR

This paper introduces HMAW, a hierarchical multi-agent workflow that enables LLMs to autonomously generate effective prompts, improving response quality without human intervention across diverse tasks.

Contribution

The paper proposes a novel, task-agnostic hierarchical framework allowing LLMs to self-design prompts, eliminating the need for human-crafted prompts or training.

Findings

HMAW improves LLM response quality across multiple benchmarks.

The approach generates detailed, task-specific prompts autonomously.

HMAW outperforms baseline prompt strategies in experiments.

Abstract

Large language models (LLMs) have shown great progress in responding to user questions, allowing for a multitude of diverse applications. Yet, the quality of LLM outputs heavily depends on the prompt design, where a good prompt might enable the LLM to answer a very challenging question correctly. Therefore, recent works have developed many strategies for improving the prompt, including both manual crafting and in-domain optimization. However, their efficacy in unrestricted scenarios remains questionable, as the former depends on human design for specific questions and the latter usually generalizes poorly to unseen scenarios. To address these problems, we give LLMs the freedom to design the best prompts according to themselves. Specifically, we include a hierarchy of LLMs, first constructing a prompt with precise instructions and accurate wording in a hierarchical manner, and then using this prompt to generate the final answer to the user query. We term this pipeline Hierarchical Multi-Agent Workflow, or HMAW. In contrast with prior works, HMAW imposes no human restriction and requires no training, and is completely task-agnostic while capable of adjusting to the nuances of the underlying task. Through both quantitative and qualitative experiments across multiple benchmarks, we verify that despite its simplicity, the proposed approach can create detailed and suitable prompts, further boosting the performance of current LLMs.