GIFT: Generalizing Intent for Flexible Test-Time Rewards

TL;DR

GIFT introduces a framework that leverages language models to infer human intent from demonstrations, enabling reward functions to generalize to new environments and objects without retraining.

Contribution

GIFT is the first method to ground reward generalization in human intent using language models, improving robustness across distribution shifts in robotic tasks.

Findings

Outperforms visual and semantic similarity baselines in simulated tasks

Achieves reliable transfer to real-world robotic manipulation

Demonstrates robustness across over 50 unseen objects

Abstract

Robots learn reward functions from user demonstrations, but these rewards often fail to generalize to new environments. This failure occurs because learned rewards latch onto spurious correlations in training data rather than the underlying human intent that demonstrations represent. Existing methods leverage visual or semantic similarity to improve robustness, yet these surface-level cues often diverge from what humans actually care about. We present Generalizing Intent for Flexible Test-Time Rewards (GIFT), a framework that grounds reward generalization in human intent rather than surface cues. GIFT leverages language models to infer high-level intent from user demonstrations by contrasting preferred with non-preferred behaviors. At deployment, GIFT maps novel test states to behaviorally equivalent training states via intent-conditioned similarity, enabling learned rewards to generalize across distribution shifts without retraining. We evaluate GIFT on tabletop manipulation tasks with new objects and layouts. Across four simulated tasks with over 50 unseen objects, GIFT consistently outperforms visual and semantic similarity baselines in test-time pairwise win rate and state-alignment F1 score. Real-world experiments on a 7-DoF Franka Panda robot demonstrate that GIFT reliably transfers to physical settings. Further discussion can be found at https://mit-clear-lab.github.io/GIFT/