Flow Dynamics Correction for Action Recognition

TL;DR

This paper introduces a flow dynamics correction method for optical flow in action recognition, boosting model performance by normalizing motion features and integrating them into existing models, achieving state-of-the-art results.

Contribution

It proposes a novel flow dynamics correction technique using power normalization and a simple hallucination step to enhance optical flow features for action recognition.

Findings

Performance improved on HMDB-51, YUP++, MPII Cooking Activities, and Charades datasets.

Corrected optical flow boosts accuracy of existing models.

Achieved state-of-the-art results with the proposed method.

Abstract

Various research studies indicate that action recognition performance highly depends on the types of motions being extracted and how accurate the human actions are represented. In this paper, we investigate different optical flow, and features extracted from these optical flow that capturing both short-term and long-term motion dynamics. We perform power normalization on the magnitude component of optical flow for flow dynamics correction to boost subtle or dampen sudden motions. We show that existing action recognition models which rely on optical flow are able to get performance boosted with our corrected optical flow. To further improve performance, we integrate our corrected flow dynamics into popular models through a simple hallucination step by selecting only the best performing optical flow features, and we show that by 'translating' the CNN feature maps into these optical flow features with different scales of motions leads to the new state-of-the-art performance on several benchmarks including HMDB-51, YUP++, fine-grained action recognition on MPII Cooking Activities, and large-scale Charades.