Large Language Model-based Human-Agent Collaboration for Complex Task Solving

TL;DR

This paper introduces ReHAC, a reinforcement learning method for optimizing human intervention in LLM-based agents to enhance performance in complex tasks, demonstrating significant improvements through strategic collaboration.

Contribution

The paper presents a novel reinforcement learning approach, ReHAC, for determining optimal human intervention points in LLM-based agents during complex task solving.

Findings

ReHAC effectively identifies when human intervention improves task performance.

Synergistic human-agent collaboration significantly outperforms autonomous approaches.

Limited, well-timed human input enhances complex task-solving efficiency.

Abstract

In recent developments within the research community, the integration of Large Language Models (LLMs) in creating fully autonomous agents has garnered significant interest. Despite this, LLM-based agents frequently demonstrate notable shortcomings in adjusting to dynamic environments and fully grasping human needs. In this work, we introduce the problem of LLM-based human-agent collaboration for complex task-solving, exploring their synergistic potential. In addition, we propose a Reinforcement Learning-based Human-Agent Collaboration method, ReHAC. This approach includes a policy model designed to determine the most opportune stages for human intervention within the task-solving process. We construct a human-agent collaboration dataset to train this policy model in an offline reinforcement learning environment. Our validation tests confirm the model's effectiveness. The results demonstrate that the synergistic efforts of humans and LLM-based agents significantly improve performance in complex tasks, primarily through well-planned, limited human intervention. Datasets and code are available at: https://github.com/XueyangFeng/ReHAC.