Spatial Parsing and Dynamic Temporal Pooling networks for Human-Object Interaction detection

TL;DR

This paper introduces SPDTP, a novel spatio-temporal graph network with explicit spatial parsing and a learnable temporal module, significantly improving video human-object interaction detection performance.

Contribution

The paper proposes a new SPDTP network with explicit spatial parsing and a dynamic temporal module for better video HOI detection, achieving state-of-the-art results.

Findings

SPDTP outperforms existing methods on CAD-120 and Something-Else datasets.

Explicit spatial parsing improves interaction discrimination.

Dynamic temporal pooling emphasizes keyframes and reduces redundancy.

Abstract

The key of Human-Object Interaction(HOI) recognition is to infer the relationship between human and objects. Recently, the image's Human-Object Interaction(HOI) detection has made significant progress. However, there is still room for improvement in video HOI detection performance. Existing one-stage methods use well-designed end-to-end networks to detect a video segment and directly predict an interaction. It makes the model learning and further optimization of the network more complex. This paper introduces the Spatial Parsing and Dynamic Temporal Pooling (SPDTP) network, which takes the entire video as a spatio-temporal graph with human and object nodes as input. Unlike existing methods, our proposed network predicts the difference between interactive and non-interactive pairs through explicit spatial parsing, and then performs interaction recognition. Moreover, we propose a learnable and differentiable Dynamic Temporal Module(DTM) to emphasize the keyframes of the video and suppress the redundant frame. Furthermore, the experimental results show that SPDTP can pay more attention to active human-object pairs and valid keyframes. Overall, we achieve state-of-the-art performance on CAD-120 dataset and Something-Else dataset.