Activity Recognition based on a Magnitude-Orientation Stream Network

TL;DR

This paper introduces a novel Magnitude-Orientation Stream (MOS) for activity recognition in videos, which enhances motion feature learning by using optical flow magnitude and orientation images, improving existing two-stream CNN methods.

Contribution

The paper proposes a new temporal stream based on optical flow magnitude and orientation, improving activity recognition accuracy over traditional methods.

Findings

MOS improves activity recognition accuracy on HMDB51 and UCF101 datasets.

Using MOS as input enhances existing neural network architectures.

The approach provides complementary motion information to classical two-stream methods.

Abstract

The temporal component of videos provides an important clue for activity recognition, as a number of activities can be reliably recognized based on the motion information. In view of that, this work proposes a novel temporal stream for two-stream convolutional networks based on images computed from the optical flow magnitude and orientation, named Magnitude-Orientation Stream (MOS), to learn the motion in a better and richer manner. Our method applies simple nonlinear transformations on the vertical and horizontal components of the optical flow to generate input images for the temporal stream. Experimental results, carried on two well-known datasets (HMDB51 and UCF101), demonstrate that using our proposed temporal stream as input to existing neural network architectures can improve their performance for activity recognition. Results demonstrate that our temporal stream provides complementary information able to improve the classical two-stream methods, indicating the suitability of our approach to be used as a temporal video representation.