CoDeTT: A Context-Aware Decision Benchmark for Turn-Taking Evaluation

TL;DR

CoDeTT introduces a comprehensive, context-aware benchmark dataset and evaluation protocol for turn-taking in dialogue systems, enabling systematic comparison across diverse scenarios.

Contribution

It formulates turn-taking as a structured decision problem and provides a multi-scenario dataset with fine-grained categories for improved evaluation.

Findings

Existing models show performance disparities across decision types.

CoDeTT enables systematic evaluation across varied interaction scenarios.

The benchmark dataset and toolkit are publicly available.

Abstract

Turn-taking modeling is fundamental to spoken dialogue systems, yet its evaluation remains fragmented and often limited to binary boundary detection under narrow interaction settings. Such protocols hinder systematic comparison and obscure model weaknesses across conversational conditions. We present CoDeTT, a context-aware decision benchmark for turn-taking evaluation. CoDeTT formulates turn-taking as a structured decision problem and constructs a multi-scenario dataset with fine-grained decision categories and controlled context variations. Under a unified evaluation protocol, we assess representative existing models and observe substantial performance disparities across decision types and interaction scenarios. CoDeTT provides a standardized benchmark for systematic and context-aware evaluation of turn-taking systems. The benchmark dataset and evaluation toolkit are available at https://yingaowang-casia.github.io/CoDeTT.github.io/.