GROWL: Group Detection With Link Prediction

TL;DR

GROWL introduces a GNN-based method for interaction group detection that predicts links between individuals using their spatial relationships, outperforming previous pairwise affinity approaches across multiple camera perspectives.

Contribution

This paper presents GROWL, a novel GNN-based approach for group detection that leverages link prediction and multimodal data, improving accuracy over existing methods.

Findings

GROWL outperforms state-of-the-art methods in accuracy.

GNN-based approach effectively handles different camera views.

Multimodal RGB and depth data enhance detection performance.

Abstract

Interaction group detection has been previously addressed with bottom-up approaches which relied on the position and orientation information of individuals. These approaches were primarily based on pairwise affinity matrices and were limited to static, third-person views. This problem can greatly benefit from a holistic approach based on Graph Neural Networks (GNNs) beyond pairwise relationships, due to the inherent spatial configuration that exists between individuals who form interaction groups. Our proposed method, GROup detection With Link prediction (GROWL), demonstrates the effectiveness of a GNN based approach. GROWL predicts the link between two individuals by generating a feature embedding based on their neighbourhood in the graph and determines whether they are connected with a shallow binary classification method such as Multi-layer Perceptrons (MLPs). We test our method against other state-of-the-art group detection approaches on both a third-person view dataset and a robocentric (i.e., egocentric) dataset. In addition, we propose a multimodal approach based on RGB and depth data to calculate a representation GROWL can utilise as input. Our results show that a GNN based approach can significantly improve accuracy across different camera views, i.e., third-person and egocentric views.