Human Interaction Recognition Framework based on Interacting Body Part Attention

TL;DR

This paper introduces a novel human interaction recognition framework that uses body part attention and combines local interaction details with overall appearance changes, improving accuracy on public datasets.

Contribution

It proposes a new framework that fuses local body part interactions with global appearance changes using attention mechanisms and LSTM for temporal modeling.

Findings

Outperforms state-of-the-art methods on four public datasets

Effectively captures subtle differences in human interactions

Utilizes body part attention to enhance feature representation

Abstract

Human activity recognition in videos has been widely studied and has recently gained significant advances with deep learning approaches; however, it remains a challenging task. In this paper, we propose a novel framework that simultaneously considers both implicit and explicit representations of human interactions by fusing information of local image where the interaction actively occurred, primitive motion with the posture of individual subject's body parts, and the co-occurrence of overall appearance change. Human interactions change, depending on how the body parts of each human interact with the other. The proposed method captures the subtle difference between different interactions using interacting body part attention. Semantically important body parts that interact with other objects are given more weight during feature representation. The combined feature of interacting body part attention-based individual representation and the co-occurrence descriptor of the full-body appearance change is fed into long short-term memory to model the temporal dynamics over time in a single framework. We validate the effectiveness of the proposed method using four widely used public datasets by outperforming the competing state-of-the-art method.