Doodle Your Keypoints: Sketch-Based Few-Shot Keypoint Detection
Subhajit Maity, Ayan Kumar Bhunia, Subhadeep Koley, Pinaki Nath Chowdhury, Aneeshan Sain, Yi-Zhe Song

TL;DR
This paper introduces a novel sketch-based framework for few-shot keypoint detection that overcomes cross-modal and style variation challenges, enabling effective learning with minimal data.
Contribution
It proposes a prototypical, grid-based domain adaptation method that leverages sketches as a source-free modality for few-shot keypoint detection.
Findings
Effective in few-shot convergence across new keypoints and classes
Achieves robust cross-modal keypoint detection
Demonstrates superior performance through extensive experiments
Abstract
Keypoint detection, integral to modern machine perception, faces challenges in few-shot learning, particularly when source data from the same distribution as the query is unavailable. This gap is addressed by leveraging sketches, a popular form of human expression, providing a source-free alternative. However, challenges arise in mastering cross-modal embeddings and handling user-specific sketch styles. Our proposed framework overcomes these hurdles with a prototypical setup, combined with a grid-based locator and prototypical domain adaptation. We also demonstrate success in few-shot convergence across novel keypoints and classes through extensive experiments.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Advanced Image and Video Retrieval Techniques · Domain Adaptation and Few-Shot Learning
