Part-Aware Bottom-Up Group Reasoning for Fine-Grained Social Interaction Detection

TL;DR

This paper introduces a part-aware bottom-up reasoning framework that improves fine-grained social interaction detection by leveraging body part cues and interpersonal relations, outperforming previous methods.

Contribution

It presents a novel approach that explicitly models nuanced social cues and local interactions for more accurate group detection in social scenarios.

Findings

Achieves state-of-the-art performance on NVI dataset.

Demonstrates generalizability on Cafe9 dataset.

Outperforms prior methods in social interaction detection.

Abstract

Social interactions often emerge from subtle, fine-grained cues such as facial expressions, gaze, and gestures. However, existing methods for social interaction detection overlook such nuanced cues and primarily rely on holistic representations of individuals. Moreover, they directly detect social groups without explicitly modeling the underlying interactions between individuals. These drawbacks limit their ability to capture localized social signals and introduce ambiguity when group configurations should be inferred from social interactions grounded in nuanced cues. In this work, we propose a part-aware bottom-up group reasoning framework for fine-grained social interaction detection. The proposed method infers social groups and their interactions using body part features and their interpersonal relations. Our model first detects individuals and enhances their features using part-aware cues, and then infers group configuration by associating individuals via similarity-based reasoning, which considers not only spatial relations but also subtle social cues that signal interactions, leading to more accurate group inference. Experiments on the NVI dataset demonstrate that our method outperforms prior methods, achieving the new state of the art, while additional results on the Caf\'e dataset further validate its generalizability to group activity understanding.