Triple-S: A Collaborative Multi-LLM Framework for Solving Long-Horizon Implicative Tasks in Robotics

TL;DR

Triple-S is a collaborative multi-LLM framework that improves the success and robustness of long-horizon robotic tasks by using roles, in-context learning, and a demonstration update mechanism, validated in simulation and real-world settings.

Contribution

It introduces a novel multi-LLM collaborative framework with role-based in-context learning and demonstration updates for robotic task execution.

Findings

Achieved 89% success rate on LDIP dataset tasks.

Improved robustness in both observable and partially observable scenarios.

Validated effectiveness in simulation and real-world robot experiments.

Abstract

Leveraging Large Language Models (LLMs) to write policy code for controlling robots has gained significant attention. However, in long-horizon implicative tasks, this approach often results in API parameter, comments and sequencing errors, leading to task failure. To address this problem, we propose a collaborative Triple-S framework that involves multiple LLMs. Through In-Context Learning, different LLMs assume specific roles in a closed-loop Simplification-Solution-Summary process, effectively improving success rates and robustness in long-horizon implicative tasks. Additionally, a novel demonstration library update mechanism which learned from success allows it to generalize to previously failed tasks. We validate the framework in the Long-horizon Desktop Implicative Placement (LDIP) dataset across various baseline models, where Triple-S successfully executes 89% of tasks in both observable and partially observable scenarios. Experiments in both simulation and real-world robot settings further validated the effectiveness of Triple-S. Our code and dataset is available at: https://github.com/Ghbbbbb/Triple-S.