Efficiently Generating Expressive Quadruped Behaviors via Language-Guided Preference Learning

TL;DR

This paper presents LGPL, a novel method combining pre-trained language models and preference learning to efficiently generate expressive quadruped behaviors aligned with human expectations, requiring fewer interactions.

Contribution

Introducing LGPL, which leverages LLM priors to guide preference learning, significantly improving sample efficiency in generating expressive robot behaviors.

Findings

LGPL learns behaviors with as few as four queries.

Outperforms purely language-based and traditional preference methods.

Achieves rapid, accurate, and expressive quadruped behaviors.

Abstract

Expressive robotic behavior is essential for the widespread acceptance of robots in social environments. Recent advancements in learned legged locomotion controllers have enabled more dynamic and versatile robot behaviors. However, determining the optimal behavior for interactions with different users across varied scenarios remains a challenge. Current methods either rely on natural language input, which is efficient but low-resolution, or learn from human preferences, which, although high-resolution, is sample inefficient. This paper introduces a novel approach that leverages priors generated by pre-trained LLMs alongside the precision of preference learning. Our method, termed Language-Guided Preference Learning (LGPL), uses LLMs to generate initial behavior samples, which are then refined through preference-based feedback to learn behaviors that closely align with human expectations. Our core insight is that LLMs can guide the sampling process for preference learning, leading to a substantial improvement in sample efficiency. We demonstrate that LGPL can quickly learn accurate and expressive behaviors with as few as four queries, outperforming both purely language-parameterized models and traditional preference learning approaches. Website with videos: https://lgpl-gaits.github.io/