Learning Skills to Patch Plans Based on Inaccurate Models

TL;DR

This paper introduces a hybrid approach that combines model-based planning with local model-free policies to improve task success and reduce data requirements when models are inaccurate.

Contribution

The method patches sub-optimal model-based planners with local learned policies only where model failures occur, enhancing reliability and efficiency.

Findings

Outperforms pure planning in reliability.

Requires less data than pure imitation learning.

Effective in shape insertion and door opening tasks.

Abstract

Planners using accurate models can be effective for accomplishing manipulation tasks in the real world, but are typically highly specialized and require significant fine-tuning to be reliable. Meanwhile, learning is useful for adaptation, but can require a substantial amount of data collection. In this paper, we propose a method that improves the efficiency of sub-optimal planners with approximate but simple and fast models by switching to a model-free policy when unexpected transitions are observed. Unlike previous work, our method specifically addresses when the planner fails due to transition model error by patching with a local policy only where needed. First, we use a sub-optimal model-based planner to perform a task until model failure is detected. Next, we learn a local model-free policy from expert demonstrations to complete the task in regions where the model failed. To show the efficacy of our method, we perform experiments with a shape insertion puzzle and compare our results to both pure planning and imitation learning approaches. We then apply our method to a door opening task. Our experiments demonstrate that our patch-enhanced planner performs more reliably than pure planning and with lower overall sample complexity than pure imitation learning.