Multi-agent Attentional Activity Recognition

TL;DR

This paper introduces a multi-agent spatial-temporal attention model for sensor-based activity recognition, effectively capturing feature salience and body part relations to improve recognition accuracy.

Contribution

It proposes a novel multi-agent attention framework that models activity-specific motions and feature selection, advancing the state-of-the-art in multi-modality activity recognition.

Findings

Outperforms existing methods on four real-world datasets

Effectively captures spatial-temporal feature salience

Models relations between activities and body part motions

Abstract

Multi-modality is an important feature of sensor based activity recognition. In this work, we consider two inherent characteristics of human activities, the spatially-temporally varying salience of features and the relations between activities and corresponding body part motions. Based on these, we propose a multi-agent spatial-temporal attention model. The spatial-temporal attention mechanism helps intelligently select informative modalities and their active periods. And the multiple agents in the proposed model represent activities with collective motions across body parts by independently selecting modalities associated with single motions. With a joint recognition goal, the agents share gained information and coordinate their selection policies to learn the optimal recognition model. The experimental results on four real-world datasets demonstrate that the proposed model outperforms the state-of-the-art methods.