TUSK: Task-Agnostic Unsupervised Keypoints
Yuhe Jin, Weiwei Sun, Jan Hosang, Eduard Trulls, Kwang Moo Yi

TL;DR
TUSK introduces a novel unsupervised keypoint learning method that can handle multiple instances and diverse tasks by using a single heatmap and clustering, broadening applicability beyond previous methods.
Contribution
The paper proposes a task-agnostic, unsupervised keypoint learning approach that handles multiple instances through clustering, unlike prior methods relying on single-instance assumptions.
Findings
Achieves state-of-the-art performance on multiple tasks
Handles multiple instances effectively in unsupervised settings
Demonstrates broad applicability across various vision tasks
Abstract
Existing unsupervised methods for keypoint learning rely heavily on the assumption that a specific keypoint type (e.g. elbow, digit, abstract geometric shape) appears only once in an image. This greatly limits their applicability, as each instance must be isolated before applying the method-an issue that is never discussed or evaluated. We thus propose a novel method to learn Task-agnostic, UnSupervised Keypoints (TUSK) which can deal with multiple instances. To achieve this, instead of the commonly-used strategy of detecting multiple heatmaps, each dedicated to a specific keypoint type, we use a single heatmap for detection, and enable unsupervised learning of keypoint types through clustering. Specifically, we encode semantics into the keypoints by teaching them to reconstruct images from a sparse set of keypoints and their descriptors, where the descriptors are forced to form…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsAdvanced Image and Video Retrieval Techniques · Multimodal Machine Learning Applications · Handwritten Text Recognition Techniques
MethodsHeatmap
