# Tied Hidden Factors in Neural Networks for End-to-End Speaker Recognition

## TL;DR

This paper introduces a neural network-based end-to-end speaker verification system that models speaker and session variability using tied hidden factors, enabling likelihood ratio computation and MAP adaptation.

## Contribution

It presents a novel neural network architecture with tied hidden variables for speaker/session variability, integrating likelihood ratio scoring and MAP adaptation in end-to-end speaker recognition.

## Key findings

- Effective modeling of speaker and session variability.
- Likelihood ratio scoring with neural networks.
- MAP adaptation during enrollment.

## Abstract

In this paper we propose a method to model speaker and session variability and able to generate likelihood ratios using neural networks in an end-to-end phrase dependent speaker verification system. As in Joint Factor Analysis, the model uses tied hidden variables to model speaker and session variability and a MAP adaptation of some of the parameters of the model. In the training procedure our method jointly estimates the network parameters and the values of the speaker and channel hidden variables. This is done in a two-step backpropagation algorithm, first the network weights and factor loading matrices are updated and then the hidden variables, whose gradients are calculated by aggregating the corresponding speaker or session frames, since these hidden variables are tied. The last layer of the network is defined as a linear regression probabilistic model whose inputs are the previous layer outputs. This choice has the advantage that it produces likelihoods and additionally it can be adapted during the enrolment using MAP without the need of a gradient optimization. The decisions are made based on the ratio of the output likelihoods of two neural network models, speaker adapted and universal background model. The method was evaluated on the RSR2015 database.