Long-term Task-oriented Agent: Proactive Long-term Intent Maintenance in Dynamic Environments

TL;DR

This paper introduces a proactive approach for task-oriented agents that maintain long-term user intents and adapt to dynamic environments through intent monitoring and event-triggered follow-ups, supported by a new benchmark.

Contribution

It proposes a novel proactive interaction paradigm, a data synthesis pipeline, and a benchmark for evaluating long-term task-oriented agents in dynamic settings.

Findings

Fine-tuned synthetic data model achieves 85.19% task completion rate.

Proposed methods reveal flaws in existing models for long-term interaction.

Benchmark enables systematic evaluation of dynamic environment handling.

Abstract

Current large language model agents predominantly operate under a reactive paradigm, responding only to immediate user queries within short-term sessions. This limitation hinders their ability to maintain long-term user's intents and dynamically adapt to evolving external environments. In this paper, we propose a novel interaction paradigm for proactive Task-oriented Agents capable of bridging the gap between relatively static user's needs and a dynamic environment. We formalize proactivity through two key capabilities, (i) Intent-Conditioned Monitoring: The agent autonomously formulates trigger conditions based on dialog history; (ii) Event-Triggered Follow-up: The agent actively engages the user upon detecting useful environmental updates. We introduce a high-quality data synthesis pipeline to construct complex, multi-turn dialog data in a dynamic environment. Furthermore, we attempt to address the lack of evaluation criteria of task-oriented interaction in a dynamic environment by proposing a new benchmark, namely ChronosBench. We evaluated some leading close-source and open-source models at present and revealed their flaws in long-term task-oriented interaction. Furthermore, our fine-tuned model trained using synthetic data for supervised learning achieves a task completion rate of 85.19% for complex tasks including shifts in user intent, outperforming other models under test. And the result validated the effectiveness of our data-driven strategy.