AVR: Synergizing Foundation Models for Audio-Visual Humor Detection

TL;DR

This paper introduces AVR, a novel audio-visual humor detection system that leverages multimodal cues without relying on textual transcriptions, addressing limitations of traditional text-based methods.

Contribution

AVR is the first system to detect humor using combined audio-visual cues without dependence on ASR or textual analysis.

Findings

Effective multimodal humor detection achieved

Eliminates reliance on ASR systems

Improves robustness in real-world scenarios

Abstract

In this work, we present, AVR application for audio-visual humor detection. While humor detection has traditionally centered around textual analysis, recent advancements have spotlighted multimodal approaches. However, these methods lean on textual cues as a modality, necessitating the use of ASR systems for transcribing the audio-data. This heavy reliance on ASR accuracy can pose challenges in real-world applications. To address this bottleneck, we propose an innovative audio-visual humor detection system that circumvents textual reliance, eliminating the need for ASR models. Instead, the proposed approach hinges on the intricate interplay between audio and visual content for effective humor detection.