MuSe-Toolbox: The Multimodal Sentiment Analysis Continuous Annotation Fusion and Discrete Class Transformation Toolbox

TL;DR

MuSe-Toolbox is an open-source Python toolkit that unifies multimodal emotion annotation fusion methods, introduces a novel alignment technique, and enables transformation of continuous emotion signals into meaningful discrete classes.

Contribution

It introduces the Rater Aligned Annotation Weighting (RAAW) method and provides the first comprehensive toolkit for continuous to discrete emotion annotation transformation.

Findings

Promising class formations outperform fixed boundaries

Effective alignment improves annotation fusion

Toolkit is easily deployable via Docker

Abstract

We introduce the MuSe-Toolbox - a Python-based open-source toolkit for creating a variety of continuous and discrete emotion gold standards. In a single framework, we unify a wide range of fusion methods and propose the novel Rater Aligned Annotation Weighting (RAAW), which aligns the annotations in a translation-invariant way before weighting and fusing them based on the inter-rater agreements between the annotations. Furthermore, discrete categories tend to be easier for humans to interpret than continuous signals. With this in mind, the MuSe-Toolbox provides the functionality to run exhaustive searches for meaningful class clusters in the continuous gold standards. To our knowledge, this is the first toolkit that provides a wide selection of state-of-the-art emotional gold standard methods and their transformation to discrete classes. Experimental results indicate that MuSe-Toolbox can provide promising and novel class formations which can be better predicted than hard-coded classes boundaries with minimal human intervention. The implementation (1) is out-of-the-box available with all dependencies using a Docker container (2).