Latent Embeddings for Collective Activity Recognition

TL;DR

This paper introduces a deep learning approach with latent embeddings and attention mechanisms to improve collective activity recognition, capturing complex interactions among individuals more effectively than previous methods.

Contribution

It proposes a novel deep learning framework that embeds latent variables with attention to model complex person-group interactions in collective activity recognition.

Findings

Outperforms state-of-the-art methods on three datasets.

Introduces a larger dataset for collective activity recognition.

Achieves more accurate recognition by modeling richer contextual information.

Abstract

Rather than simply recognizing the action of a person individually, collective activity recognition aims to find out what a group of people is acting in a collective scene. Previ- ous state-of-the-art methods using hand-crafted potentials in conventional graphical model which can only define a limited range of relations. Thus, the complex structural de- pendencies among individuals involved in a collective sce- nario cannot be fully modeled. In this paper, we overcome these limitations by embedding latent variables into feature space and learning the feature mapping functions in a deep learning framework. The embeddings of latent variables build a global relation containing person-group interac- tions and richer contextual information by jointly modeling broader range of individuals. Besides, we assemble atten- tion mechanism during embedding for achieving more com- pact representations. We evaluate our method on three col- lective activity datasets, where we contribute a much larger dataset in this work. The proposed model has achieved clearly better performance as compared to the state-of-the- art methods in our experiments.