Animal Pose Labeling Using General-Purpose Point Trackers
Zhuoyang Pan, Boxiao Pan, Guandao Yang, Adam W. Harley, Leonidas Guibas

TL;DR
This paper introduces a novel test time optimization approach for animal pose labeling in videos, fine-tuning a general-purpose point tracker with sparse annotations to achieve state-of-the-art results efficiently.
Contribution
It proposes a new pipeline that uses test time optimization and sparse annotations to improve animal pose labeling accuracy and efficiency.
Findings
Achieves state-of-the-art performance on animal pose labeling tasks.
Reduces annotation costs compared to existing methods.
Provides a practical tool for automatic animal behavior analysis.
Abstract
Automatically estimating animal poses from videos is important for studying animal behaviors. Existing methods do not perform reliably since they are trained on datasets that are not comprehensive enough to capture all necessary animal behaviors. However, it is very challenging to collect such datasets due to the large variations in animal morphology. In this paper, we propose an animal pose labeling pipeline that follows a different strategy, i.e. test time optimization. Given a video, we fine-tune a lightweight appearance embedding inside a pre-trained general-purpose point tracker on a sparse set of annotated frames. These annotations can be obtained from human labelers or off-the-shelf pose detectors. The fine-tuned model is then applied to the rest of the frames for automatic labeling. Our method achieves state-of-the-art performance at a reasonable annotation cost. We believe our…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsZebrafish Biomedical Research Applications · Human Motion and Animation · Human Pose and Action Recognition
