Body-Part Joint Detection and Association via Extended Object Representation

TL;DR

This paper introduces BPJDet, a unified, anchor-based detector that jointly detects human bodies and parts, embedding semantic and geometric info to improve association accuracy and speed, validated on multiple datasets.

Contribution

Proposes a novel extended object representation and a dense single-stage detector for joint body-part detection and association.

Findings

Achieves state-of-the-art association performance on CityPersons, CrowdHuman, and BodyHands datasets.

Maintains high detection accuracy while improving association accuracy.

Does not require error-prone post-matching for body-part association.

Abstract

The detection of human body and its related parts (e.g., face, head or hands) have been intensively studied and greatly improved since the breakthrough of deep CNNs. However, most of these detectors are trained independently, making it a challenging task to associate detected body parts with people. This paper focuses on the problem of joint detection of human body and its corresponding parts. Specifically, we propose a novel extended object representation that integrates the center location offsets of body or its parts, and construct a dense single-stage anchor-based Body-Part Joint Detector (BPJDet). Body-part associations in BPJDet are embedded into the unified representation which contains both the semantic and geometric information. Therefore, BPJDet does not suffer from error-prone association post-matching, and has a better accuracy-speed trade-off. Furthermore, BPJDet can be seamlessly generalized to jointly detect any body part. To verify the effectiveness and superiority of our method, we conduct extensive experiments on the CityPersons, CrowdHuman and BodyHands datasets. The proposed BPJDet detector achieves state-of-the-art association performance on these three benchmarks while maintains high accuracy of detection. Code is in https://github.com/hnuzhy/BPJDet.