Effective Task Planning with Missing Objects using Learning-Informed Object Search

TL;DR

This paper introduces a novel planning framework for mobile robots that integrates learning-driven object search actions into task planning, enabling effective operation despite unknown object locations.

Contribution

It develops model-based LIOS actions and a high-level planning approach that interleaves search and execution, improving task success under uncertainty.

Findings

Outperforms baseline methods in simulated environments.

Effective in real-world retrieval and meal prep tasks.

Maintains compatibility with existing full-knowledge solvers.

Abstract

Task planning for mobile robots often assumes full environment knowledge and so popular approaches, like planning via the PDDL, cannot plan when the locations of task-critical objects are unknown. Recent learning-driven object search approaches are effective, but operate as standalone tools and so are not straightforwardly incorporated into full task planners, which must additionally determine both what objects are necessary and when in the plan they should be sought out. To address this limitation, we develop a planning framework centered around novel model-based LIOS actions: each a policy that aims to find and retrieve a single object. High-level planning treats LIOS actions as deterministic and so -- informed by model-based calculations of the expected cost of each -- generates plans that interleave search and execution for effective, sound, and complete learning-informed task planning despite uncertainty. Our work effectively reasons about uncertainty while maintaining compatibility with existing full-knowledge solvers. In simulated ProcTHOR homes and in the real world, our approach outperforms non-learned and learned baselines on tasks including retrieval and meal prep.