Generative Model-Based Feature Attention Module for Video Action Analysis

TL;DR

This paper introduces a generative attention-based model that enhances video action analysis by effectively learning feature semantics, improving accuracy in action recognition and detection tasks for IoT applications.

Contribution

The paper presents a novel generative attention module that captures feature semantics and temporal dependencies, addressing limitations of existing methods in IoT-oriented video analysis.

Findings

Outperforms existing methods on benchmark datasets

Improves accuracy in action recognition and detection

Effectively learns feature semantics from foreground and background differences

Abstract

Video action analysis is a foundational technology within the realm of intelligent video comprehension, particularly concerning its application in Internet of Things(IoT). However, existing methodologies overlook feature semantics in feature extraction and focus on optimizing action proposals, thus these solutions are unsuitable for widespread adoption in high-performance IoT applications due to the limitations in precision, such as autonomous driving, which necessitate robust and scalable intelligent video analytics analysis. To address this issue, we propose a novel generative attention-based model to learn the relation of feature semantics. Specifically, by leveraging the differences of actions' foreground and background, our model simultaneously learns the frame- and segment-dependencies of temporal action feature semantics, which takes advantage of feature semantics in the feature extraction effectively. To evaluate the effectiveness of our model, we conduct extensive experiments on two benchmark video task, action recognition and action detection. In the context of action detection tasks, we substantiate the superiority of our approach through comprehensive validation on widely recognized datasets. Moreover, we extend the validation of the effectiveness of our proposed method to a broader task, video action recognition. Our code is available at https://github.com/Generative-Feature-Model/GAF.