Beyond Task-Oriented and Chitchat Dialogues: Proactive and Transition-Aware Conversational Agents

TL;DR

This paper introduces TACT, a dataset for transition-aware dialogue modeling, and demonstrates that models trained on it with DPO improve in handling mode switches and response quality in complex conversations.

Contribution

The paper presents TACT, a novel dataset for transition-aware dialogue modeling, and shows that training models with DPO on TACT enhances transition handling and response quality.

Findings

Models trained on TACT outperform baselines in intent detection.

DPO applied to TACT-trained models improves mode transition accuracy.

Achieved 75.74% joint mode-intent accuracy and 70.1% win rate in human evaluation.

Abstract

Conversational agents have traditionally been developed for either task-oriented dialogue (TOD) or open-ended chitchat, with limited progress in unifying the two. Yet, real-world conversations naturally involve fluid transitions between these modes. To address this gap, we introduce TACT (TOD-And-Chitchat Transition), a dataset designed for transition-aware dialogue modeling that incorporates structurally diverse and integrated mode flows. TACT supports both user- and agent-driven mode switches, enabling robust modeling of complex conversational dynamics. To evaluate an agent's ability to initiate and recover from mode transitions, we propose two new metrics -- Switch and Recovery. Models trained on TACT outperform baselines in both intent detection and mode transition handling. Moreover, applying Direct Preference Optimization (DPO) to TACT-trained models yields additional gains, achieving 75.74\% joint mode-intent accuracy and a 70.1\% win rate against GPT-4o in human evaluation. These results demonstrate that pairing structurally diverse data with DPO enhances response quality and transition control, paving the way for more proactive and transition-aware conversational agents.