Embodied Learning of Reward for Musculoskeletal Control with Vision Language Models

TL;DR

This paper presents MoVLR, a framework that uses vision-language models to automatically discover and refine reward functions for complex musculoskeletal control tasks, bridging high-level goals and low-level control strategies.

Contribution

Introduces MoVLR, a novel method leveraging vision-language models to iteratively learn reward functions from natural language and visual feedback for high-dimensional motor control.

Findings

Effective reward discovery for musculoskeletal control

Aligns control policies with natural language goals

Enables structured guidance for embodied learning

Abstract

Discovering effective reward functions remains a fundamental challenge in motor control of high-dimensional musculoskeletal systems. While humans can describe movement goals explicitly such as "walking forward with an upright posture," the underlying control strategies that realize these goals are largely implicit, making it difficult to directly design rewards from high-level goals and natural language descriptions. We introduce Motion from Vision-Language Representation (MoVLR), a framework that leverages vision-language models (VLMs) to bridge the gap between goal specification and movement control. Rather than relying on handcrafted rewards, MoVLR iteratively explores the reward space through iterative interaction between control optimization and VLM feedback, aligning control policies with physically coordinated behaviors. Our approach transforms language and vision-based assessments into structured guidance for embodied learning, enabling the discovery and refinement of reward functions for high-dimensional musculoskeletal locomotion and manipulation. These results suggest that VLMs can effectively ground abstract motion descriptions in the implicit principles governing physiological motor control.