Steering Robots with Inference-Time Interactions

TL;DR

This paper introduces a method to make pretrained robot policies steerable at inference time, allowing user interactions to correct errors and guide behavior without additional training.

Contribution

It proposes inference-time steering techniques that enable user-guided correction and editing of robot behaviors in pretrained policies without finetuning.

Findings

Enables switching between discrete skills via user interactions.

Allows editing of continuous motions to satisfy task constraints.

Corrects policy errors without additional training.

Abstract

Imitation learning has driven the development of generalist policies capable of autonomously solving multiple tasks. However, when a pretrained policy makes errors during deployment, there are limited mechanisms for users to correct its behavior. While collecting additional data for finetuning can address such issues, doing so for each downstream use case is inefficient at deployment. My research proposes an alternative: keeping pretrained policies frozen as a fixed skill repertoire while allowing user interactions to guide behavior generation toward user preferences at inference time. By making pretrained policies steerable, users can help correct policy errors when the model struggles to generalize-without needing to finetune the policy. Specifically, I propose (1) inference-time steering, which leverages user interactions to switch between discrete skills, and (2) task and motion imitation, which enables user interactions to edit continuous motions while satisfying task constraints defined by discrete symbolic plans. These frameworks correct misaligned policy predictions without requiring additional training, maximizing the utility of pretrained models while achieving inference-time user objectives.