Learning Decoupling Features Through Orthogonality Regularization

TL;DR

This paper introduces a two-branch neural network with orthogonality regularization to simultaneously learn shared and task-specific features for keyword spotting and speaker verification, achieving state-of-the-art results.

Contribution

It proposes a novel decoupling feature learning method using orthogonality regularization for joint KWS and SV tasks.

Findings

Achieved SOTA EER of 1.31% on KWS

Achieved SOTA EER of 1.87% on SV

Effective decoupling of features improves performance

Abstract

Keyword spotting (KWS) and speaker verification (SV) are two important tasks in speech applications. Research shows that the state-of-art KWS and SV models are trained independently using different datasets since they expect to learn distinctive acoustic features. However, humans can distinguish language content and the speaker identity simultaneously. Motivated by this, we believe it is important to explore a method that can effectively extract common features while decoupling task-specific features. Bearing this in mind, a two-branch deep network (KWS branch and SV branch) with the same network structure is developed and a novel decoupling feature learning method is proposed to push up the performance of KWS and SV simultaneously where speaker-invariant keyword representations and keyword-invariant speaker representations are expected respectively. Experiments are conducted on Google Speech Commands Dataset (GSCD). The results demonstrate that the orthogonality regularization helps the network to achieve SOTA EER of 1.31% and 1.87% on KWS and SV, respectively.