Mixtac: A Novel Bio-Inspired Hybrid Tactile Sensor with Synergistic Event-Frame Perception

TL;DR

Mixtac is a bio-inspired hybrid tactile sensor that combines event-based and frame-based data to achieve high-frequency force estimation with long-term accuracy, mimicking biological mechanoreceptors.

Contribution

The paper introduces Mixtac, a novel hybrid tactile sensor with a new neural network architecture (FGER-Net) for synergistic data fusion, addressing limitations of existing sensors.

Findings

Achieved an MAE of 0.04 N in force estimation.

Bridged the sampling rate gap from 0 to 500 Hz in tactile sensing.

Demonstrated improved long-term static force estimation.

Abstract

Vision based and event based tactile sensors are important in robotic manipulation research. However, they suffer from a fundamental tradeoff: vision based sensors have low sampling rates, while event based sensors are prone to drift during long term static force estimation. To solve this challenge and achieve human level tactile perception, the novel hybrid event frame tactile sensor (Mixtac) is proposed in this paper by emulating the synergistic function of biological mechanoreceptors, which achieves normal force estimation. The prototype leverages events for high frequency force tracking and frames for long term accuracy. The Frame Guided Event Recurrent Network (FGER-Net) was proposed to fuse the two data streams. Frames were used by the net to correct event drift during training and guide high frequency predictions during inference. Experiments demonstrated an MAE of 0.04 N. This paper could bridge the sampling rate gap from 0 to 500 Hz in current vision based tactile sensors and pave the way for human level robotic manipulation.