PatchGraph: In-hand tactile tracking with learned surface normals
Paloma Sodhi, Michael Kaess, Mustafa Mukadam, Stuart Anderson

TL;DR
PatchGraph introduces a novel tactile tracking method that leverages learned surface normals from tactile images to accurately track small objects during in-hand manipulation without prior object information.
Contribution
The paper presents a two-stage approach combining learned surface normals and factor graphs for online local patch reconstruction and object pose estimation, removing the need for prior object models.
Findings
Reliable tracking over 100 contact sequences
Effective in simulation and real-world scenarios
Works across multiple object shapes
Abstract
We address the problem of tracking 3D object poses from touch during in-hand manipulations. Specifically, we look at tracking small objects using vision-based tactile sensors that provide high-dimensional tactile image measurements at the point of contact. While prior work has relied on a-priori information about the object being localized, we remove this requirement. Our key insight is that an object is composed of several local surface patches, each informative enough to achieve reliable object tracking. Moreover, we can recover the geometry of this local patch online by extracting local surface normal information embedded in each tactile image. We propose a novel two-stage approach. First, we learn a mapping from tactile images to surface normals using an image translation network. Second, we use these surface normals within a factor graph to both reconstruct a local patch map and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTactile and Sensory Interactions · Robot Manipulation and Learning · Muscle activation and electromyography studies
