TrackDeform3D: Markerless and Autonomous 3D Keypoint Tracking and Dataset Collection for Deformable Objects
Yeheng Zong, Yizhou Chen, Alexander Bowler, Chia-Tung Yang, Ram Vasudevan

TL;DR
This paper introduces TrackDeform3D, an autonomous RGB-D based system for collecting large-scale 3D deformable object datasets with accurate keypoint tracking, addressing the scarcity of such data for research.
Contribution
It presents a novel, affordable framework for autonomous 3D data collection of deformable objects, improving keypoint tracking accuracy and enabling large-scale dataset creation.
Findings
TrackDeform3D outperforms state-of-the-art methods in accuracy.
The framework successfully collects 110 minutes of high-quality deformable object data.
The dataset includes diverse object categories and trajectories.
Abstract
Structured 3D representations such as keypoints and meshes offer compact, expressive descriptions of deformable objects, jointly capturing geometric and topological information useful for downstream tasks such as dynamics modeling and motion planning. However, robustly extracting such representations remains challenging, as current perception methods struggle to handle complex deformations. Moreover, large-scale 3D data collection remains a bottleneck: existing approaches either require prohibitive data collection efforts, such as labor-intensive annotation or expensive motion capture setups, or rely on simplifying assumptions that break down in unstructured environments. As a result, large-scale 3D datasets and benchmarks for deformable objects remain scarce. To address these challenges, this paper presents an affordable and autonomous framework for collecting 3D datasets of deformable…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRobotics and Sensor-Based Localization · 3D Shape Modeling and Analysis · Advanced Vision and Imaging
