Activity Recognition on a Large Scale in Short Videos - Moments in Time Dataset

TL;DR

This paper advances activity recognition in short videos by leveraging multimodal features and novel fusion techniques, achieving high accuracy on the Moments in Time dataset.

Contribution

It introduces a new approach combining visual and textual features with fusion methods for improved activity classification.

Findings

Achieved 89.23% Top-5 accuracy on 20 classes

Outperformed baseline TRN model significantly

Demonstrated effectiveness of visual-textual fusion techniques

Abstract

Moments capture a huge part of our lives. Accurate recognition of these moments is challenging due to the diverse and complex interpretation of the moments. Action recognition refers to the act of classifying the desired action/activity present in a given video. In this work, we perform experiments on Moments in Time dataset to recognize accurately activities occurring in 3 second clips. We use state of the art techniques for visual, auditory and spatio temporal localization and develop method to accurately classify the activity in the Moments in Time dataset. Our novel approach of using Visual Based Textual features and fusion techniques performs well providing an overall 89.23 % Top - 5 accuracy on the 20 classes - a significant improvement over the Baseline TRN model.