When to Laugh and How Hard? A Multimodal Approach to Detecting Humor and its Intensity

TL;DR

This paper introduces a multimodal model that automatically detects humor and its intensity in TV show dialogues by analyzing prerecorded laughter, achieving high accuracy in humor detection and laughter duration prediction.

Contribution

It presents a novel approach using multimodal data and prerecorded laughter as annotations to detect humor and measure its intensity in TV shows.

Findings

Humor detection accuracy of 78%

Laughter duration prediction with a mean absolute error of 600 ms

Effective use of prerecorded laughter as humor annotation

Abstract

Prerecorded laughter accompanying dialog in comedy TV shows encourages the audience to laugh by clearly marking humorous moments in the show. We present an approach for automatically detecting humor in the Friends TV show using multimodal data. Our model is capable of recognizing whether an utterance is humorous or not and assess the intensity of it. We use the prerecorded laughter in the show as annotation as it marks humor and the length of the audience's laughter tells us how funny a given joke is. We evaluate the model on episodes the model has not been exposed to during the training phase. Our results show that the model is capable of correctly detecting whether an utterance is humorous 78% of the time and how long the audience's laughter reaction should last with a mean absolute error of 600 milliseconds.