Anticipate, Adapt, Act: A Hybrid Framework for Task Planning

TL;DR

This paper introduces a hybrid framework combining large language models and probabilistic decision-making to improve robot planning, anticipation, and adaptation in complex human-robot collaboration tasks, showing significant performance gains.

Contribution

It presents a novel hybrid approach integrating LLMs with influence diagrams for proactive failure prediction and recovery in robot task planning.

Findings

Significant performance improvement over baselines in VirtualHome 3D simulations.

Effective prediction of failures due to human or object limitations.

Enhanced robot adaptability in complex tasks.

Abstract

Anticipating and adapting to failures is a key capability robots need to collaborate effectively with humans in complex domains. This continues to be a challenge despite the impressive performance of state of the art AI planning systems and Large Language Models (LLMs) because of the uncertainty associated with the tasks and their outcomes. Toward addressing this challenge, we present a hybrid framework that integrates the generic prediction capabilities of an LLM with the probabilistic sequential decision-making capability of Relational Dynamic Influence Diagram Language. For any given task, the robot reasons about the task and the capabilities of the human attempting to complete it; predicts potential failures due to lack of ability (in the human) or lack of relevant domain objects; and executes actions to prevent such failures or recover from them. Experimental evaluation in the VirtualHome 3D simulation environment demonstrates substantial improvement in performance compared with state of the art baselines.