# Knowledge Augmented Relation Inference for Group Activity Recognition

## TL;DR

This paper introduces a novel framework that leverages concretized knowledge to enhance relation inference and individual representations, significantly improving group activity recognition accuracy.

## Contribution

It proposes a Knowledge Augmented Relation Inference framework that integrates visual and semantic knowledge for better group activity recognition.

## Key findings

- Achieves competitive performance on public datasets.
- Effectively utilizes knowledge to improve relation inference.
- Enhances individual representations with knowledge integration.

## Abstract

Most existing group activity recognition methods construct spatial-temporal relations merely based on visual representation. Some methods introduce extra knowledge, such as action labels, to build semantic relations and use them to refine the visual presentation. However, the knowledge they explored just stay at the semantic-level, which is insufficient for pursing notable accuracy. In this paper, we propose to exploit knowledge concretization for the group activity recognition, and develop a novel Knowledge Augmented Relation Inference framework that can effectively use the concretized knowledge to improve the individual representations. Specifically, the framework consists of a Visual Representation Module to extract individual appearance features, a Knowledge Augmented Semantic Relation Module explore semantic representations of individual actions, and a Knowledge-Semantic-Visual Interaction Module aims to integrate visual and semantic information by the knowledge. Benefiting from these modules, the proposed framework can utilize knowledge to enhance the relation inference process and the individual representations, thus improving the performance of group activity recognition. Experimental results on two public datasets show that the proposed framework achieves competitive performance compared with state-of-the-art methods.