From Actions to Events: A Transfer Learning Approach Using Improved Deep Belief Networks

TL;DR

This paper introduces a transfer learning method using Spectral Deep Belief Networks to improve video event recognition, reducing computational costs while capturing spatial and temporal information across frames.

Contribution

It presents a novel energy-based model that transfers knowledge from action to event recognition, processing all frames simultaneously for better efficiency.

Findings

Effective on HMDB-51 and UCF-101 datasets

Reduces computational burden compared to traditional models

Captures spatial and temporal information simultaneously

Abstract

In the last decade, exponential data growth supplied machine learning-based algorithms' capacity and enabled their usage in daily-life activities. Additionally, such an improvement is partially explained due to the advent of deep learning techniques, i.e., stacks of simple architectures that end up in more complex models. Although both factors produce outstanding results, they also pose drawbacks regarding the learning process as training complex models over large datasets are expensive and time-consuming. Such a problem is even more evident when dealing with video analysis. Some works have considered transfer learning or domain adaptation, i.e., approaches that map the knowledge from one domain to another, to ease the training burden, yet most of them operate over individual or small blocks of frames. This paper proposes a novel approach to map the knowledge from action recognition to event recognition using an energy-based model, denoted as Spectral Deep Belief Network. Such a model can process all frames simultaneously, carrying spatial and temporal information through the learning process. The experimental results conducted over two public video dataset, the HMDB-51 and the UCF-101, depict the effectiveness of the proposed model and its reduced computational burden when compared to traditional energy-based models, such as Restricted Boltzmann Machines and Deep Belief Networks.