MultiLinguahah : A New Unsupervised Multilingual Acoustic Laughter Segmentation Method

TL;DR

This paper introduces MultiLinguahah, an unsupervised multilingual laughter segmentation method that outperforms existing algorithms in non-English audio contexts by using anomaly detection on energy-based audio representations.

Contribution

The paper presents a novel unsupervised approach for multilingual laughter segmentation using anomaly detection with energy-based features and BYOL-A, addressing limitations of prior English-centric datasets.

Findings

Outperforms state-of-the-art laughter detection algorithms in non-English datasets.

Uses an Isolation Forest on BYOL-A audio representations for anomaly detection.

Effective across diverse audio sources like comedy, sitcoms, and general audio.

Abstract

Laughter is a social non-vocalization that is universal across cultures and languages, and is crucial for human communication, including social bonding and communication signaling. However, detecting laughter in audio is a challenging task, and segmenting is even more difficult. Currently, Machine Learning methods generally rely on costly manual annotation, and their datasets are mostly based on English contexts. Thus, we propose an unsupervised multilingual method that sets up the laughter segmentation task as an anomaly detection of energy-based segmented audio sequences. Our method applies an Isolation Forest on audio representations learned from BYOL-A encoder. We compare our method with several state-of-the-art laughter detection algorithms on four datasets, including stand-up comedy, sitcoms, and general short audio from AudioSet. Our results show that state-of-the-art methods are not optimized for multilingual contexts, while our method outperforms them in non-English settings.