Deep Spatiotemporal Models for Robust Proprioceptive Terrain Classification

TL;DR

This paper introduces a deep LSTM-based model utilizing vehicle-terrain interaction sounds for robust proprioceptive terrain classification, achieving state-of-the-art results and improved noise robustness for autonomous robots in diverse environments.

Contribution

It presents a novel CNN-LSTM architecture for terrain classification using sound, addressing accuracy and robustness issues of prior proprioceptive methods.

Findings

State-of-the-art accuracy on two large datasets

Robust performance in high ambient noise conditions

Effective noise-aware training scheme for real-world deployment

Abstract

Terrain classification is a critical component of any autonomous mobile robot system operating in unknown real-world environments. Over the years, several proprioceptive terrain classification techniques have been introduced to increase robustness or act as a fallback for traditional vision based approaches. However, they lack widespread adaptation due to various factors that include inadequate accuracy, robustness and slow run-times. In this paper, we use vehicle-terrain interaction sounds as a proprioceptive modality and propose a deep Long-Short Term Memory (LSTM) based recurrent model that captures both the spatial and temporal dynamics of such a problem, thereby overcoming these past limitations. Our model consists of a new Convolution Neural Network (CNN) architecture that learns deep spatial features, complemented with LSTM units that learn complex temporal dynamics. Experiments on two extensive datasets collected with different microphones on various indoor and outdoor terrains demonstrate state-of-the-art performance compared to existing techniques. We additionally evaluate the performance in adverse acoustic conditions with high ambient noise and propose a noise-aware training scheme that enables learning of more generalizable models that are essential for robust real-world deployments.