Task-Centric Policy Optimization from Misaligned Motion Priors

TL;DR

This paper introduces Task-Centric Motion Priors (TCMP), a novel imitation learning framework that adaptively integrates motion priors with task objectives, improving humanoid control robustness despite misaligned demonstrations.

Contribution

The paper proposes TCMP, a task-priority adversarial imitation method that selectively incorporates motion priors based on task compatibility, addressing limitations of linear reward mixing.

Findings

TCMP achieves robust humanoid control with natural motion styles.

Theoretical analysis explains gradient conflicts and stationary points.

Experimental results show improved task performance under noisy demonstrations.

Abstract

Humanoid control often leverages motion priors from human demonstrations to encourage natural behaviors. However, such demonstrations are frequently suboptimal or misaligned with robotic tasks due to embodiment differences, retargeting errors, and task-irrelevant variations, causing na\"ive imitation to degrade task performance. Conversely, task-only reinforcement learning admits many task-optimal solutions, often resulting in unnatural or unstable motions. This exposes a fundamental limitation of linear reward mixing in adversarial imitation learning. We propose \emph{Task-Centric Motion Priors} (TCMP), a task-priority adversarial imitation framework that treats imitation as a conditional regularizer rather than a co-equal objective. TCMP maximizes task improvement while incorporating imitation signals only when they are compatible with task progress, yielding an adaptive, geometry-aware update that preserves task-feasible descent and suppresses harmful imitation under misalignment. We provide theoretical analysis of gradient conflict and task-priority stationary points, and validate our claims through humanoid control experiments demonstrating robust task performance with consistent motion style under noisy demonstrations.