LMPNet for Weakly-supervised Keypoint Discovery

TL;DR

LMPNet introduces a weakly-supervised approach for semantic object keypoint discovery using a novel leaky max pooling layer to encourage meaningful filter activations, achieving accuracy comparable to supervised methods.

Contribution

The paper proposes LMPNet, a novel weakly-supervised method that transforms intermediate filters into keypoint detectors without requiring detailed annotations.

Findings

LMPNet automatically discovers semantic keypoints.

It achieves prediction accuracy comparable to supervised models.

The method is robust to object pose variations.

Abstract

In this work, we explore the task of semantic object keypoint discovery weakly-supervised by only category labels. This is achieved by transforming discriminatively-trained intermediate layer filters into keypoint detectors. We begin by identifying three preferred characteristics of keypoint detectors: (i) spatially sparse activations, (ii) consistency and (iii) diversity. Instead of relying on hand-crafted loss terms, a novel computationally-efficient leaky max pooling (LMP) layer is proposed to explicitly encourage final conv-layer filters to learn "non-repeatable local patterns" that are well aligned with object keypoints. Informed by visualizations, a simple yet effective selection strategy is proposed to ensure consistent filter activations and attention mask-out is then applied to force the network to distribute its attention to the whole object instead of just the most discriminative region. For the final keypoint prediction, a learnable clustering layer is proposed to group keypoint proposals into keypoint predictions. The final model, named LMPNet, is highly interpretable in that it directly manipulates network filters to detect predefined concepts. Our experiments show that LMPNet can (i) automatically discover semantic keypoints that are robust to object pose and (ii) achieves strong prediction accuracy comparable to a supervised pose estimation model.