In-Context Iterative Policy Improvement for Dynamic Manipulation

TL;DR

This paper introduces an iterative in-context learning approach for dynamic manipulation tasks using pre-trained language models, demonstrating superior performance in low-data scenarios on simulation and real robot experiments.

Contribution

It presents a novel iterative in-context learning method for dynamic manipulation, addressing challenges like high dimensionality and partial observability with pre-trained language models.

Findings

Outperforms alternative methods in simulation and on physical robots

Effective in low-data regimes for complex manipulation tasks

Demonstrates the applicability of language models to robotics

Abstract

Attention-based architectures trained on internet-scale language data have demonstrated state of the art reasoning ability for various language-based tasks, such as logic problems and textual reasoning. Additionally, these Large Language Models (LLMs) have exhibited the ability to perform few-shot prediction via in-context learning, in which input-output examples provided in the prompt are generalized to new inputs. This ability furthermore extends beyond standard language tasks, enabling few-shot learning for general patterns. In this work, we consider the application of in-context learning with pre-trained language models for dynamic manipulation. Dynamic manipulation introduces several crucial challenges, including increased dimensionality, complex dynamics, and partial observability. To address this, we take an iterative approach, and formulate our in-context learning problem to predict adjustments to a parametric policy based on previous interactions. We show across several tasks in simulation and on a physical robot that utilizing in-context learning outperforms alternative methods in the low data regime. Video summary of this work and experiments can be found https://youtu.be/2inxpdrq74U?si=dAdDYsUEr25nZvRn.