PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images
Patrick Grady, Jeremy A. Collins, Chengcheng Tang, Christopher D., Twigg, Kunal Aneja, James Hays, Charles C. Kemp

TL;DR
PressureVision++ is a novel deep learning approach that estimates fingertip pressure from RGB images using a new dataset and weak supervision, enabling diverse, real-world applications like touch-sensitive interfaces.
Contribution
The paper introduces PressureVision++, a method that leverages weak labels and a new dataset to accurately estimate fingertip pressure from RGB images in diverse conditions.
Findings
PressureVision++ outperforms human annotators and prior models.
The dataset includes 51 participants with diverse objects.
Application demonstrated in mixed reality interfaces.
Abstract
Touch plays a fundamental role in manipulation for humans; however, machine perception of contact and pressure typically requires invasive sensors. Recent research has shown that deep models can estimate hand pressure based on a single RGB image. However, evaluations have been limited to controlled settings since collecting diverse data with ground-truth pressure measurements is difficult. We present a novel approach that enables diverse data to be captured with only an RGB camera and a cooperative participant. Our key insight is that people can be prompted to apply pressure in a certain way, and this prompt can serve as a weak label to supervise models to perform well under varied conditions. We collect a novel dataset with 51 participants making fingertip contact with diverse objects. Our network, PressureVision++, outperforms human annotators and prior work. We also demonstrate an…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
PressureVision++: Estimating Fingertip Pressure From Diverse RGB Images· youtube
Taxonomy
TopicsTactile and Sensory Interactions · Gaze Tracking and Assistive Technology · Muscle activation and electromyography studies
