TransTouch: Learning Transparent Objects Depth Sensing Through Sparse Touches
Liuyu Bian, Pengyang Shi, Weihang Chen, Jing Xu, Li Yi, Rui Chen

TL;DR
This paper introduces TransTouch, a method that uses tactile feedback to fine-tune stereo depth networks for transparent objects, significantly improving real-world sensing accuracy with minimal data collection.
Contribution
It proposes a novel tactile-guided finetuning approach with an optimized probing strategy and confidence regularization for transparent object depth sensing.
Findings
Enhanced depth sensing accuracy for transparent objects
Effective tactile-guided finetuning reduces data requirements
Constructed a real-world dataset for evaluation
Abstract
Transparent objects are common in daily life. However, depth sensing for transparent objects remains a challenging problem. While learning-based methods can leverage shape priors to improve the sensing quality, the labor-intensive data collection in the real world and the sim-to-real domain gap restrict these methods' scalability. In this paper, we propose a method to finetune a stereo network with sparse depth labels automatically collected using a probing system with tactile feedback. We present a novel utility function to evaluate the benefit of touches. By approximating and optimizing the utility function, we can optimize the probing locations given a fixed touching budget to better improve the network's performance on real objects. We further combine tactile depth supervision with a confidence-based regularization to prevent over-fitting during finetuning. To evaluate the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTactile and Sensory Interactions · Interactive and Immersive Displays · Augmented Reality Applications
