Salsa as a Nonverbal Embodied Language -- The CoMPAS3D Dataset and Benchmarks

TL;DR

This paper introduces CoMPAS3D, a comprehensive salsa dance dataset with expert annotations, and benchmarks for developing expressive, interactive humanoid AI capable of partner dance communication and coordination.

Contribution

It provides the largest, most diverse salsa motion dataset with detailed annotations and establishes benchmark tasks for expressive humanoid dance AI research.

Findings

First dataset with fine-grained salsa annotations

Benchmark tasks for dance partner communication

Baseline models demonstrating task feasibility

Abstract

Imagine a humanoid that can safely and creatively dance with a human, adapting to its partner's proficiency, using haptic signaling as a primary form of communication. While today's AI systems excel at text or voice-based interaction with large language models, human communication extends far beyond text-it includes embodied movement, timing, and physical coordination. Modeling coupled interaction between two agents poses a formidable challenge: it is continuous, bidirectionally reactive, and shaped by individual variation. We present CoMPAS3D, the largest and most diverse motion capture dataset of improvised salsa dancing, designed as a challenging testbed for interactive, expressive humanoid AI. The dataset includes 3 hours of leader-follower salsa dances performed by 18 dancers spanning beginner, intermediate, and professional skill levels. For the first time, we provide fine-grained salsa expert annotations, covering over 2,800 move segments, including move types, combinations, execution errors and stylistic elements. We draw analogies between partner dance communication and natural language, evaluating CoMPAS3D on two benchmark tasks for synthetic humans that parallel key problems in spoken language and dialogue processing: leader or follower generation with proficiency levels (speaker or listener synthesis), and duet (conversation) generation. Towards a long-term goal of partner dance with humans, we release the dataset, annotations, and code, along with a multitask SalsaAgent model capable of performing all benchmark tasks, alongside additional baselines to encourage research in socially interactive embodied AI and creative, expressive humanoid motion generation.