In-Materia Speech Recognition

TL;DR

This paper introduces an in-materia edge speech recognition system combining analogue feature extraction and in-memory neural network classification, achieving high accuracy with ultra-low power consumption suitable for edge devices.

Contribution

It presents a novel in-materia computing hardware architecture integrating a dopant-network-processing-unit and memristive crossbar arrays for efficient, low-power speech recognition at the edge.

Findings

Achieved 96.2% accuracy on TI-46-Word speech recognition task.

DNPU feature extraction consumes only hundreds of nanowatts.

AIMC classification potentially uses less than 10 femtojoules per operation.

Abstract

With the rise of decentralized computing, as in the Internet of Things, autonomous driving, and personalized healthcare, it is increasingly important to process time-dependent signals at the edge efficiently: right at the place where the temporal data are collected, avoiding time-consuming, insecure, and costly communication with a centralized computing facility (or cloud). However, modern-day processors often cannot meet the restrained power and time budgets of edge systems because of intrinsic limitations imposed by their architecture (von Neumann bottleneck) or domain conversions (analogue-to-digital and time-to-frequency). Here, we propose an edge temporal-signal processor based on two in-materia computing systems for both feature extraction and classification, reaching a software-level accuracy of 96.2% for the TI-46-Word speech-recognition task. First, a nonlinear, room-temperature dopant-network-processing-unit (DNPU) layer realizes analogue, time-domain feature extraction from the raw audio signals, similar to the human cochlea. Second, an analogue in-memory computing (AIMC) chip, consisting of memristive crossbar arrays, implements a compact neural network trained on the extracted features for classification. With the DNPU feature extraction consuming 100s nW and AIMC-based classification having the potential for less than 10 fJ per multiply-accumulate operation, our findings offer a promising avenue for advancing the compactness, efficiency, and performance of heterogeneous smart edge processors through in-materia computing hardware.