Conditional Link Prediction of Category-Implicit Keypoint Detection

TL;DR

This paper introduces KLPNet, an end-to-end network for simultaneous multi-class keypoint detection and conditional link prediction, improving efficiency and occlusion handling in object skeleton modeling.

Contribution

The paper presents the first unified approach for multi-class keypoint detection and category-dependent link prediction, with novel modules for feature aggregation and conditional graph modeling.

Findings

Outperforms state-of-the-art methods on three benchmarks.

Effectively handles occlusion in keypoint and link prediction.

Achieves accurate multi-class keypoint localization.

Abstract

Keypoints of objects reflect their concise abstractions, while the corresponding connection links (CL) build the skeleton by detecting the intrinsic relations between keypoints. Existing approaches are typically computationally-intensive, inapplicable for instances belonging to multiple classes, and/or infeasible to simultaneously encode connection information. To address the aforementioned issues, we propose an end-to-end category-implicit Keypoint and Link Prediction Network (KLPNet), which is the first approach for simultaneous semantic keypoint detection (for multi-class instances) and CL rejuvenation. In our KLPNet, a novel Conditional Link Prediction Graph is proposed for link prediction among keypoints that are contingent on a predefined category. Furthermore, a Cross-stage Keypoint Localization Module (CKLM) is introduced to explore feature aggregation for coarse-to-fine keypoint localization. Comprehensive experiments conducted on three publicly available benchmarks demonstrate that our KLPNet consistently outperforms all other state-of-the-art approaches. Furthermore, the experimental results of CL prediction also show the effectiveness of our KLPNet with respect to occlusion problems.