Noise-Conditioned Mixture-of-Experts Framework for Robust Speaker Verification

TL;DR

This paper introduces a noise-conditioned mixture-of-experts framework for speaker verification that improves robustness under noisy conditions by routing inputs to specialized experts based on noise characteristics.

Contribution

It proposes a novel noise-conditioned expert routing mechanism and a curriculum learning protocol, enhancing robustness and generalization in noisy environments.

Findings

Outperforms baseline methods in noisy speaker verification tasks

Demonstrates improved robustness across diverse noise conditions

Effective noise-aware expert specialization enhances speaker identity preservation

Abstract

Robust speaker verification under noisy conditions remains an open challenge. Conventional deep learning methods learn a robust unified speaker representation space against diverse background noise and achieve significant improvement. In contrast, this paper presents a noise-conditioned mixture-ofexperts framework that decomposes the feature space into specialized noise-aware subspaces for speaker verification. Specifically, we propose a noise-conditioned expert routing mechanism, a universal model based expert specialization strategy, and an SNR-decaying curriculum learning protocol, collectively improving model robustness and generalization under diverse noise conditions. The proposed method can automatically route inputs to expert networks based on noise information derived from the inputs, where each expert targets distinct noise characteristics while preserving speaker identity information. Comprehensive experiments demonstrate consistent superiority over baselines