DiariZen Explained: A Tutorial for the Open Source State-of-the-Art Speaker Diarization Pipeline

TL;DR

This paper provides a comprehensive, step-by-step tutorial explaining the open-source DiariZen speaker diarization pipeline, detailing each component and offering source code and visualizations for better understanding and reproducibility.

Contribution

It offers a self-contained, detailed walkthrough of the DiariZen pipeline, making it easier for researchers to understand, reproduce, and extend this state-of-the-art system.

Findings

DiariZen achieves leading performance across multiple benchmarks.

The tutorial includes source code, intermediate visualizations, and end-to-end execution scripts.

The pipeline integrates WavLM, Conformer, and VBx clustering for speaker diarization.

Abstract

Speaker diarization (SD) is the task of answering "who spoke when" in a multi-speaker audio stream. Classically, an SD system clusters segments of speech belonging to an individual speaker's identity. Recent years have seen substantial progress in SD through end-to-end neural diarization (EEND) approaches. DiariZen, a hybrid SD pipeline built upon a structurally pruned WavLM-Large encoder, a Conformer backend with powerset classification, and VBx clustering, represents the leading open-source state of the art at the time of writing across multiple benchmarks. Despite its strong performance, the DiariZen architecture spans several repositories and frameworks, making it difficult for researchers and practitioners to understand, reproduce, or extend the system as a whole. This tutorial paper provides a self-contained, block-by-block explanation of the complete DiariZen pipeline, decomposing it into seven stages: (1) audio loading and sliding window segmentation, (2) WavLM feature extraction with learned layer weighting, (3) Conformer backend and powerset classification, (4) segmentation aggregation via overlap-add, (5) speaker embedding extraction with overlap exclusion, (6) VBx clustering with PLDA scoring, and (7) reconstruction and RTTM output. For each block, we provide the conceptual motivation, source code references, intermediate tensor shapes, and annotated visualizations of the actual outputs on a 30s excerpt from the AMI Meeting Corpus. The implementation is available at https://github.com/nikhilraghav29/diarizen-tutorial, which includes standalone executable scripts for each block and a Jupyter notebook that runs the complete pipeline end-to-end.