RR-Net: Injecting Interactive Semantics in Human-Object Interaction Detection

TL;DR

RR-Net introduces relation reasoning modules that enhance human-object interaction detection by explicitly modeling interactive semantics, leading to state-of-the-art performance on standard benchmarks.

Contribution

The paper proposes a novel relation reasoning framework with new modules for interactive semantics, significantly improving HOI detection accuracy.

Findings

Sets new state-of-the-art on V-COCO and HICO-DET benchmarks.

Improves baseline performance by approximately 5.5% and 9.8%.

Demonstrates the effectiveness of relation reasoning in end-to-end HOI detection.

Abstract

Human-Object Interaction (HOI) detection devotes to learn how humans interact with surrounding objects. Latest end-to-end HOI detectors are short of relation reasoning, which leads to inability to learn HOI-specific interactive semantics for predictions. In this paper, we therefore propose novel relation reasoning for HOI detection. We first present a progressive Relation-aware Frame, which brings a new structure and parameter sharing pattern for interaction inference. Upon the frame, an Interaction Intensifier Module and a Correlation Parsing Module are carefully designed, where: a) interactive semantics from humans can be exploited and passed to objects to intensify interactions, b) interactive correlations among humans, objects and interactions are integrated to promote predictions. Based on modules above, we construct an end-to-end trainable framework named Relation Reasoning Network (abbr. RR-Net). Extensive experiments show that our proposed RR-Net sets a new state-of-the-art on both V-COCO and HICO-DET benchmarks and improves the baseline about 5.5% and 9.8% relatively, validating that this first effort in exploring relation reasoning and integrating interactive semantics has brought obvious improvement for end-to-end HOI detection.