ExPO: Explainable Phonetic Trait-Oriented Network for Speaker Verification

TL;DR

This paper introduces ExPO, an explainable speaker verification system that models phonetic traits to provide interpretable results similar to forensic voice comparison, and evaluates the most effective phonetic traits for verification.

Contribution

The paper presents a novel phonetic trait-oriented network that offers explainability in speaker verification and analyzes phonetic traits for improved accuracy.

Findings

ExPO enables fine-grained phonetic trait analysis.

Phonetic traits significantly impact verification accuracy.

The system provides visualizations for interpretability.

Abstract

In speaker verification, we use computational method to verify if an utterance matches the identity of an enrolled speaker. This task is similar to the manual task of forensic voice comparison, where linguistic analysis is combined with auditory measurements to compare and evaluate voice samples. Despite much success, we have yet to develop a speaker verification system that offers explainable results comparable to those from manual forensic voice comparison. A novel approach, Explainable Phonetic Trait-Oriented (ExPO) network, is proposed in this paper to introduce the speaker's phonetic trait which describes the speaker's characteristics at the phonetic level, resembling what forensic comparison does. ExPO not only generates utterance-level speaker embeddings but also allows for fine-grained analysis and visualization of phonetic traits, offering an explainable speaker verification process. Furthermore, we investigate phonetic traits from within-speaker and between-speaker variation perspectives to determine which trait is most effective for speaker verification, marking an important step towards explainable speaker verification. Our code is available at https://github.com/mmmmayi/ExPO.