Spatio-Temporal Dynamic Inference Network for Group Activity Recognition

TL;DR

This paper introduces a Dynamic Inference Network that models person-specific spatio-temporal interactions for group activity recognition, improving accuracy and efficiency over previous methods by dynamically predicting relations and interaction paths.

Contribution

The paper presents a novel Dynamic Inference Network with relation and walk modules that dynamically construct person-specific interaction graphs for better activity recognition.

Findings

Significant accuracy improvements over state-of-the-art methods.

Reduced computational overhead in reasoning modules.

Effective modeling of person-specific interactions.

Abstract

Group activity recognition aims to understand the activity performed by a group of people. In order to solve it, modeling complex spatio-temporal interactions is the key. Previous methods are limited in reasoning on a predefined graph, which ignores the inherent person-specific interaction context. Moreover, they adopt inference schemes that are computationally expensive and easily result in the over-smoothing problem. In this paper, we manage to achieve spatio-temporal person-specific inferences by proposing Dynamic Inference Network (DIN), which composes of Dynamic Relation (DR) module and Dynamic Walk (DW) module. We firstly propose to initialize interaction fields on a primary spatio-temporal graph. Within each interaction field, we apply DR to predict the relation matrix and DW to predict the dynamic walk offsets in a joint-processing manner, thus forming a person-specific interaction graph. By updating features on the specific graph, a person can possess a global-level interaction field with a local initialization. Experiments indicate both modules' effectiveness. Moreover, DIN achieves significant improvement compared to previous state-of-the-art methods on two popular datasets under the same setting, while costing much less computation overhead of the reasoning module.