MULTI-Bench: A Multi-Turn Interactive Benchmark for Assessing Emotional Intelligence ability of Spoken Dialogue Models

TL;DR

This paper introduces Multi-Bench, a comprehensive benchmark designed to evaluate spoken dialogue models' emotional intelligence and multi-turn interaction capabilities, highlighting current models' strengths and areas for improvement.

Contribution

Multi-Bench is the first benchmark specifically targeting multi-turn emotional intelligence in spoken dialogue models, with a hierarchical structure and diverse tasks to assess understanding and application.

Findings

Current SDMs perform well on basic emotion understanding tasks.

Models show room for improvement in complex reasoning and emotional application.

Evaluation on six models reveals strengths and gaps in emotional intelligence capabilities.

Abstract

Spoken Dialogue Models (SDMs) have advanced rapidly, yet their ability to sustain genuinely interactive multi-turn conversations remains underexplored, as most benchmarks focus on single-turn exchanges. We introduce Multi-Bench, the first benchmark explicitly designed to evaluate SDMs in multi-turn interactive dialogue with an emphasis on emotional intelligence. Multi-Bench employs a hierarchical structure with a basic track for emotion understanding and reasoning and an advanced track for emotion support and application. It comprises five carefully designed tasks and about 3.2K samples, ranging from emotion recognition to complex reasoning and interactive dialogue, supported by a reproducible evaluation framework. We evaluate six representative SDMs on eight subsets of Multi-Bench. Results show that while current SDMs achieve good performance on basic understanding tasks, they still have room for improvement in advanced multi-turn interactive dialogue and reasoning-related tasks, particularly in emotion awareness and application.