Learning to Plan with Personalized Preferences

TL;DR

This paper introduces a new benchmark and methods for AI agents to learn and adapt to individual human preferences in planning tasks, improving personalized plan generation.

Contribution

It develops a framework for agents to learn preferences from few demonstrations and adapt their planning strategies accordingly, introducing the Preference-based Planning benchmark.

Findings

Symbol-based approaches show promise in scalability.

Significant challenges remain in learning personalized plans.

Incorporating preferences improves plan personalization.

Abstract

Effective integration of AI agents into daily life requires them to understand and adapt to individual human preferences, particularly in collaborative roles. Although recent studies on embodied intelligence have advanced significantly, they typically adopt generalized approaches that overlook personal preferences in planning. We address this limitation by developing agents that not only learn preferences from few demonstrations but also learn to adapt their planning strategies based on these preferences. Our research leverages the observation that preferences, though implicitly expressed through minimal demonstrations, can generalize across diverse planning scenarios. To systematically evaluate this hypothesis, we introduce Preference-based Planning (PbP) benchmark, an embodied benchmark featuring hundreds of diverse preferences spanning from atomic actions to complex sequences. Our evaluation of SOTA methods reveals that while symbol-based approaches show promise in scalability, significant challenges remain in learning to generate and execute plans that satisfy personalized preferences. We further demonstrate that incorporating learned preferences as intermediate representations in planning significantly improves the agent's ability to construct personalized plans. These findings establish preferences as a valuable abstraction layer for adaptive planning, opening new directions for research in preference-guided plan generation and execution.