You're Pushing My Buttons: Instrumented Learning of Gentle Button Presses
Raman Talwar, Remko Proesmans, Thomas Lips, Andreas Verleysen, Francis wyffels

TL;DR
This paper investigates using sensorized contact information during training to improve contact-rich manipulation policies, demonstrating that auxiliary contact signals can reduce contact force without affecting success rates.
Contribution
It introduces a method of sensorizing objects with microphones and privileged signals to enhance contact detection and force regulation in manipulation policies.
Findings
Instrumentation-guided audio representations reduce contact force.
Success rates are similar across methods with and without instrumentation.
Sensorized training improves contact safety without sacrificing performance.
Abstract
Learning contact-rich manipulation is difficult from cameras and proprioception alone because contact events are only partially observed. We test whether training-time instrumentation, i.e., object sensorisation, can improve policy performance without creating deployment-time dependencies. Specifically, we study button pressing as a testbed and use a microphone fingertip to capture contact-relevant audio. We use an instrumented button-state signal as privileged supervision to fine-tune an audio encoder into a contact event detector. We combine the resulting representation with imitation learning using three strategies, such that the policy only uses vision and audio during inference. Button press success rates are similar across methods, but instrumentation-guided audio representations consistently reduce contact force. These results support instrumentation as a practical training-time…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
