Temporal Activity Detection in Untrimmed Videos with Recurrent Neural Networks

TL;DR

This thesis presents a neural network-based approach combining 3D CNN features and recurrent networks for classifying and localizing activities in untrimmed videos, achieving competitive results in a major challenge.

Contribution

It introduces a flexible architecture integrating 3D CNN features with RNNs for activity detection and localization in videos, with optimized post-processing techniques.

Findings

Achieved competitive results in ActivityNet Challenge 2016

Demonstrated effectiveness of combined 3D CNN and RNN architecture

Improved temporal localization through post-processing methods

Abstract

This thesis explore different approaches using Convolutional and Recurrent Neural Networks to classify and temporally localize activities on videos, furthermore an implementation to achieve it has been proposed. As the first step, features have been extracted from video frames using an state of the art 3D Convolutional Neural Network. This features are fed in a recurrent neural network that solves the activity classification and temporally location tasks in a simple and flexible way. Different architectures and configurations have been tested in order to achieve the best performance and learning of the video dataset provided. In addition it has been studied different kind of post processing over the trained network's output to achieve a better results on the temporally localization of activities on the videos. The results provided by the neural network developed in this thesis have been submitted to the ActivityNet Challenge 2016 of the CVPR, achieving competitive results using a simple and flexible architecture.