PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detection

TL;DR

This paper introduces PPDM, a real-time, single-stage HOI detection method that outperforms existing approaches in speed and accuracy by using parallel point detection and matching, enabling efficient human-object interaction recognition.

Contribution

The paper presents a novel parallel architecture for HOI detection that improves efficiency and accuracy by predicting points and matching them in a single stage, and introduces a new HOI dataset.

Findings

Achieves 37 fps on a Titan XP GPU, outperforming all existing methods.

First real-time HOI detection method.

Provides a new HOI dataset, HOI-A.

Abstract

We propose a single-stage Human-Object Interaction (HOI) detection method that has outperformed all existing methods on HICO-DET dataset at 37 fps on a single Titan XP GPU. It is the first real-time HOI detection method. Conventional HOI detection methods are composed of two stages, i.e., human-object proposals generation, and proposals classification. Their effectiveness and efficiency are limited by the sequential and separate architecture. In this paper, we propose a Parallel Point Detection and Matching (PPDM) HOI detection framework. In PPDM, an HOI is defined as a point triplet < human point, interaction point, object point>. Human and object points are the center of the detection boxes, and the interaction point is the midpoint of the human and object points. PPDM contains two parallel branches, namely point detection branch and point matching branch. The point detection branch predicts three points. Simultaneously, the point matching branch predicts two displacements from the interaction point to its corresponding human and object points. The human point and the object point originated from the same interaction point are considered as matched pairs. In our novel parallel architecture, the interaction points implicitly provide context and regularization for human and object detection. The isolated detection boxes are unlikely to form meaning HOI triplets are suppressed, which increases the precision of HOI detection. Moreover, the matching between human and object detection boxes is only applied around limited numbers of filtered candidate interaction points, which saves much computational cost. Additionally, we build a new application-oriented database named HOI-A, which severs as a good supplement to the existing datasets. The source code and the dataset will be made publicly available to facilitate the development of HOI detection.