HabitAction: A Video Dataset for Human Habitual Behavior Recognition

TL;DR

This paper introduces HabitAction, a comprehensive video dataset capturing human habitual behaviors, and proposes a two-stream model that effectively recognizes these behaviors by analyzing local features, advancing human action recognition.

Contribution

The paper presents a novel dataset for habitual behaviors and a two-stream recognition model that outperforms existing methods on this dataset.

Findings

Proposed dataset contains 30 habitual behavior categories with over 300,000 frames.

Two-stream model combining skeletons and RGB improves recognition accuracy.

Model outperforms existing methods on the HabitAction dataset.

Abstract

Human Action Recognition (HAR) is a very crucial task in computer vision. It helps to carry out a series of downstream tasks, like understanding human behaviors. Due to the complexity of human behaviors, many highly valuable behaviors are not yet encompassed within the available datasets for HAR, e.g., human habitual behaviors (HHBs). HHBs hold significant importance for analyzing a person's personality, habits, and psychological changes. To solve these problems, in this work, we build a novel video dataset to demonstrate various HHBs. These behaviors in the proposed dataset are able to reflect internal mental states and specific emotions of the characters, e.g., crossing arms suggests to shield oneself from perceived threats. The dataset contains 30 categories of habitual behaviors including more than 300,000 frames and 6,899 action instances. Since these behaviors usually appear at small local parts of human action videos, it is difficult for existing action recognition methods to handle these local features. Therefore, we also propose a two-stream model using both human skeletons and RGB appearances. Experimental results demonstrate that our proposed method has much better performance in action recognition than the existing methods on the proposed dataset.