VEMOCLAP: A video emotion classification web application

TL;DR

VEMOCLAP is an open-source web app that classifies emotions in videos using pretrained models and multi-head cross-attention, achieving state-of-the-art accuracy and allowing user video analysis.

Contribution

It introduces VEMOCLAP, the first accessible web application for video emotion classification that leverages pretrained features and multi-head cross-attention for improved accuracy.

Findings

Increased classification accuracy by 4.3% on Ekman-6 dataset.

First open-source web app for video emotion analysis.

Enables users to analyze their own or YouTube videos.

Abstract

We introduce VEMOCLAP: Video EMOtion Classifier using Pretrained features, the first readily available and open-source web application that analyzes the emotional content of any user-provided video. We improve our previous work, which exploits open-source pretrained models that work on video frames and audio, and then efficiently fuse the resulting pretrained features using multi-head cross-attention. Our approach increases the state-of-the-art classification accuracy on the Ekman-6 video emotion dataset by 4.3% and offers an online application for users to run our model on their own videos or YouTube videos. We invite the readers to try our application at serkansulun.com/app.