Towards Multimodal MIR: Predicting individual differences from music-induced movement

TL;DR

This study demonstrates that individual differences such as personality and empathy can be accurately predicted from free dance movements, advancing multimodal music information retrieval and personalized recommendation systems.

Contribution

It introduces a model that predicts personality and empathy traits from dance movements, including the novel prediction of SQ, with high accuracy.

Findings

High R2 scores for personality, EQ, and SQ predictions

Identification of key bodily joints related to traits

Potential for personalized music recommendation and therapy

Abstract

As the field of Music Information Retrieval grows, it is important to take into consideration the multi-modality of music and how aspects of musical engagement such as movement and gesture might be taken into account. Bodily movement is universally associated with music and reflective of important individual features related to music preference such as personality, mood, and empathy. Future multimodal MIR systems may benefit from taking these aspects into account. The current study addresses this by identifying individual differences, specifically Big Five personality traits, and scores on the Empathy and Systemizing Quotients (EQ/SQ) from participants' free dance movements. Our model successfully explored the unseen space for personality as well as EQ, SQ, which has not previously been accomplished for the latter. R2 scores for personality, EQ, and SQ were 76.3%, 77.1%, and 86.7% respectively. As a follow-up, we investigated which bodily joints were most important in defining these traits. We discuss how further research may explore how the mapping of these traits to movement patterns can be used to build a more personalized, multi-modal recommendation system, as well as potential therapeutic applications.