An Investigation of Multi-feature Extraction and Super-resolution with Fast Microphone Arrays

TL;DR

This paper demonstrates that a sparse array of MEMS microphones can be used as vibration sensors for fast, accurate tactile sensing tasks like texture classification and contact localization, leveraging transformer-based models on high-rate time-series data.

Contribution

The study introduces a novel use of MEMS microphone arrays combined with transformer architectures for multi-feature extraction and super-resolution tactile sensing.

Findings

Achieved 77.3% accuracy in 4-class texture classification

Obtained 1.8 mm mean error in contact localization

Demonstrated robustness to varying and unseen velocities

Abstract

In this work, we use MEMS microphones as vibration sensors to simultaneously classify texture and estimate contact position and velocity. Vibration sensors are an important facet of both human and robotic tactile sensing, providing fast detection of contact and onset of slip. Microphones are an attractive option for implementing vibration sensing as they offer a fast response and can be sampled quickly, are affordable, and occupy a very small footprint. Our prototype sensor uses only a sparse array (8-9 mm spacing) of distributed MEMS microphones (<$1, 3.76 x 2.95 x 1.10 mm) embedded under an elastomer. We use transformer-based architectures for data analysis, taking advantage of the microphones' high sampling rate to run our models on time-series data as opposed to individual snapshots. This approach allows us to obtain 77.3% average accuracy on 4-class texture classification (84.2% when excluding the slowest drag velocity), 1.8 mm mean error on contact localization, and 5.6 mm/s mean error on contact velocity. We show that the learned texture and localization models are robust to varying velocity and generalize to unseen velocities. We also report that our sensor provides fast contact detection, an important advantage of fast transducers. This investigation illustrates the capabilities one can achieve with a MEMS microphone array alone, leaving valuable sensor real estate available for integration with complementary tactile sensing modalities.