Deep Neural Object Analysis by Interactive Auditory Exploration with a Humanoid Robot

TL;DR

This paper introduces a novel method where a humanoid robot uses interactive shaking and auditory analysis with neural networks to identify object properties like material and weight, demonstrating robustness to noise.

Contribution

The paper presents a new interactive auditory analysis framework combining robotic manipulation and neural networks for object property estimation.

Findings

High accuracy in material classification and weight prediction

Robustness of the framework to real-world acoustic noise

Effective integration of robotic interaction with neural auditory analysis

Abstract

We present a novel approach for interactive auditory object analysis with a humanoid robot. The robot elicits sensory information by physically shaking visually indistinguishable plastic capsules. It gathers the resulting audio signals from microphones that are embedded into the robotic ears. A neural network architecture learns from these signals to analyze properties of the contents of the containers. Specifically, we evaluate the material classification and weight prediction accuracy and demonstrate that the framework is fairly robust to acoustic real-world noise.