Multimodal Contact Detection using Auditory and Force Features for Reliable Object Placing in Household Environments

TL;DR

This paper introduces a multimodal contact detection method combining auditory and force features, enabling reliable object placement in household environments without relying on force/torque thresholds.

Contribution

The proposed approach models contact events using frequency domain features of auditory and haptic signals, eliminating the need for threshold-based detection in dynamic settings.

Findings

Achieved 99.94% accuracy in contact detection

Effective in highly dynamic household scenarios

Independent of force/torque threshold values

Abstract

Typical contact detection is based on the monitoring of a threshold value in the force and torque signals. The selection of a threshold is challenging for robots operating in unstructured or highly dynamic environments, such in a household setting, due to the variability of the characteristics of the objects that might be encountered. We propose a multimodal contact detection approach using time and frequency domain features which model the distinctive characteristics of contact events in the auditory and haptic modalities. In our approach the monitoring of force and torque thresholds is not necessary as detection is based on the characteristics of force and torque signals in the frequency domain together with the impact sound generated by the manipulation task. We evaluated our approach with a typical glass placing task in a household setting. Our experimental results show that robust contact detection (99.94% mean cross-validation accuracy) is possible independent of force/torque threshold values and suitable of being implemented for operation in highly dynamic scenarios.