Learning Adaptive Node Selection with External Attention for Human Interaction Recognition

TL;DR

This paper introduces ASEA, a novel GCN-based framework with external attention and adaptive node selection that dynamically models human interactions, outperforming existing methods in recognizing complex social actions.

Contribution

The paper proposes a new adaptive node selection mechanism and external attention module for dynamic interaction modeling in human activity recognition.

Findings

Achieves state-of-the-art performance on benchmark datasets.

Effectively captures complex interaction relationships.

Reduces irrelevant information through adaptive node selection.

Abstract

Most GCN-based methods model interacting individuals as independent graphs, neglecting their inherent inter-dependencies. Although recent approaches utilize predefined interaction adjacency matrices to integrate participants, these matrices fail to adaptively capture the dynamic and context-specific joint interactions across different actions. In this paper, we propose the Active Node Selection with External Attention Network (ASEA), an innovative approach that dynamically captures interaction relationships without predefined assumptions. Our method models each participant individually using a GCN to capture intra-personal relationships, facilitating a detailed representation of their actions. To identify the most relevant nodes for interaction modeling, we introduce the Adaptive Temporal Node Amplitude Calculation (AT-NAC) module, which estimates global node activity by combining spatial motion magnitude with adaptive temporal weighting, thereby highlighting salient motion patterns while reducing irrelevant or redundant information. A learnable threshold, regularized to prevent extreme variations, is defined to selectively identify the most informative nodes for interaction modeling. To capture interactions, we design the External Attention (EA) module to operate on active nodes, effectively modeling the interaction dynamics and semantic relationships between individuals. Extensive evaluations show that our method captures interaction relationships more effectively and flexibly, achieving state-of-the-art performance.