Me-Agent: A Personalized Mobile Agent with Two-Level User Habit Learning for Enhanced Interaction

TL;DR

Me-Agent is a personalized mobile agent that uses a two-level user habit learning approach, combining preference learning and hierarchical memory, to better interpret ambiguous instructions and adapt to user needs.

Contribution

The paper introduces Me-Agent, a novel personalized mobile agent with a two-level habit learning framework and a new benchmark for ambiguous instructions.

Findings

Achieves state-of-the-art personalization performance

Maintains competitive instruction execution efficiency

Effectively interprets ambiguous user instructions

Abstract

Large Language Model (LLM)-based mobile agents have made significant performance advancements. However, these agents often follow explicit user instructions while overlooking personalized needs, leading to significant limitations for real users, particularly without personalized context: (1) inability to interpret ambiguous instructions, (2) lack of learning from user interaction history, and (3) failure to handle personalized instructions. To alleviate the above challenges, we propose Me-Agent, a learnable and memorable personalized mobile agent. Specifically, Me-Agent incorporates a two-level user habit learning approach. At the prompt level, we design a user preference learning strategy enhanced with a Personal Reward Model to improve personalization performance. At the memory level, we design a Hierarchical Preference Memory, which stores users' long-term memory and app-specific memory in different level memory. To validate the personalization capabilities of mobile agents, we introduce User FingerTip, a new benchmark featuring numerous ambiguous instructions for daily life. Extensive experiments on User FingerTip and general benchmarks demonstrate that Me-Agent achieves state-of-the-art performance in personalization while maintaining competitive instruction execution performance.