TinySV: Speaker Verification in TinyML with On-device Learning

TL;DR

This paper introduces TinySV, an on-device learning solution for speaker verification in TinyML, focusing on reducing resource demands and enabling adaptive learning on tiny IoT devices.

Contribution

The paper presents a novel hierarchical TinyML approach for on-device speaker verification that operates efficiently with limited data and computational resources.

Findings

Effective on real-world IoT device

Reduces memory and computational overhead

Operates with few and unlabelled training data

Abstract

TinyML is a novel area of machine learning that gained huge momentum in the last few years thanks to the ability to execute machine learning algorithms on tiny devices (such as Internet-of-Things or embedded systems). Interestingly, research in this area focused on the efficient execution of the inference phase of TinyML models on tiny devices, while very few solutions for on-device learning of TinyML models are available in the literature due to the relevant overhead introduced by the learning algorithms. The aim of this paper is to introduce a new type of adaptive TinyML solution that can be used in tasks, such as the presented \textit{Tiny Speaker Verification} (TinySV), that require to be tackled with an on-device learning algorithm. Achieving this goal required (i) reducing the memory and computational demand of TinyML learning algorithms, and (ii) designing a TinyML learning algorithm operating with few and possibly unlabelled training data. The proposed TinySV solution relies on a two-layer hierarchical TinyML solution comprising Keyword Spotting and Adaptive Speaker Verification module. We evaluated the effectiveness and efficiency of the proposed TinySV solution on a dataset collected expressly for the task and tested the proposed solution on a real-world IoT device (Infineon PSoC 62S2 Wi-Fi BT Pioneer Kit).