Spatially Conditioned Graphs for Detecting Human-Object Interactions

TL;DR

This paper introduces spatially conditioned graph neural networks for human-object interaction detection, leveraging spatial relationships to improve message passing and graph feature refinement, leading to state-of-the-art results.

Contribution

It proposes a novel spatial conditioning mechanism for message passing in graph neural networks, enhancing human-object interaction detection performance.

Findings

Spatial conditioning improves interaction detection accuracy.

The method outperforms existing approaches on HICO-DET and V-COCO datasets.

Spatial features become more important as bounding box quality increases.

Abstract

We address the problem of detecting human-object interactions in images using graphical neural networks. Unlike conventional methods, where nodes send scaled but otherwise identical messages to each of their neighbours, we propose to condition messages between pairs of nodes on their spatial relationships, resulting in different messages going to neighbours of the same node. To this end, we explore various ways of applying spatial conditioning under a multi-branch structure. Through extensive experimentation we demonstrate the advantages of spatial conditioning for the computation of the adjacency structure, messages and the refined graph features. In particular, we empirically show that as the quality of the bounding boxes increases, their coarse appearance features contribute relatively less to the disambiguation of interactions compared to the spatial information. Our method achieves an mAP of 31.33% on HICO-DET and 54.2% on V-COCO, significantly outperforming state-of-the-art on fine-tuned detections.