# Learning Embodied Semantics via Music and Dance Semiotic Correlations

## TL;DR

This paper presents a computational model that learns semiotic correlations between music and dance, demonstrating embodied cognition by effectively linking auditory and visual modalities in a cross-modal retrieval task.

## Contribution

It introduces a novel statistical model that captures embodied semantics by correlating music audio with dance video, advancing understanding of music perception.

## Key findings

- Model successfully retrieves dance videos from music queries and vice versa.
- Quantitative results show statistically significant correlation between music and dance.
- Supports embodied cognition theory in music perception.

## Abstract

Music semantics is embodied, in the sense that meaning is biologically mediated by and grounded in the human body and brain. This embodied cognition perspective also explains why music structures modulate kinetic and somatosensory perception. We leverage this aspect of cognition, by considering dance as a proxy for music perception, in a statistical computational model that learns semiotic correlations between music audio and dance video. We evaluate the ability of this model to effectively capture underlying semantics in a cross-modal retrieval task. Quantitative results, validated with statistical significance testing, strengthen the body of evidence for embodied cognition in music and show the model can recommend music audio for dance video queries and vice-versa.