Out-of-Distribution Recovery with Object-Centric Keypoint Inverse Policy for Visuomotor Imitation Learning

TL;DR

This paper introduces an object-centric recovery framework that enhances visuomotor imitation learning by guiding policies back to training distribution in out-of-distribution scenarios, improving robustness without extra data collection.

Contribution

It presents a novel object-centric recovery policy based on inverse policy inference from object keypoints, which can be added to existing policies to handle OOD situations.

Findings

77.7% improvement over base policy in OOD scenarios

Effective in both simulation and real robot experiments

Enables autonomous demonstration collection for continual learning

Abstract

We propose an object-centric recovery (OCR) framework to address the challenges of out-of-distribution (OOD) scenarios in visuomotor policy learning. Previous behavior cloning (BC) methods rely heavily on a large amount of labeled data coverage, failing in unfamiliar spatial states. Without relying on extra data collection, our approach learns a recovery policy constructed by an inverse policy inferred from the object keypoint manifold gradient in the original training data. The recovery policy serves as a simple add-on to any base visuomotor BC policy, agnostic to a specific method, guiding the system back towards the training distribution to ensure task success even in OOD situations. We demonstrate the effectiveness of our object-centric framework in both simulation and real robot experiments, achieving an improvement of 77.7\% over the base policy in OOD. Furthermore, we show OCR's capacity to autonomously collect demonstrations for continual learning. Overall, we believe this framework represents a step toward improving the robustness of visuomotor policies in real-world settings.