Controllable Conversational Theme Detection Track at DSTC 12
Igor Shalyminov, Hang Su, Jake Vincent, Siffi Singh, Jason Cai, James Gung, Raphael Shu, Saab Mansour

TL;DR
This paper introduces a new controllable theme detection task for conversational analytics, framing it as a DSTC 12 challenge that involves clustering and labeling dialog utterances with adjustable granularity based on user preferences.
Contribution
It presents a novel problem formulation for controllable conversational theme detection, along with dataset, evaluation metrics, and a public competition track to advance research in this area.
Findings
Multiple participant solutions analyzed
Controllability of theme granularity demonstrated
Open dataset and code released for future research
Abstract
Conversational analytics has been on the forefront of transformation driven by the advances in Speech and Natural Language Processing techniques. Rapid adoption of Large Language Models (LLMs) in the analytics field has taken the problems that can be automated to a new level of complexity and scale. In this paper, we introduce Theme Detection as a critical task in conversational analytics, aimed at automatically identifying and categorizing topics within conversations. This process can significantly reduce the manual effort involved in analyzing expansive dialogs, particularly in domains like customer support or sales. Unlike traditional dialog intent detection, which often relies on a fixed set of intents for downstream system logic, themes are intended as a direct, user-facing summary of the conversation's core inquiry. This distinction allows for greater flexibility in theme surface…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
