Automatic Quality Assessment for Audio-Visual Verification Systems. The LOVe submission to NIST SRE Challenge 2019

TL;DR

This paper introduces a universal quality assessment model for audio-visual verification systems, improving multimodal fusion by estimating the quality of face and speaker representations, leading to enhanced verification performance.

Contribution

A novel universal quality assessment model for both face and speaker modalities that improves score-level fusion in multimodal biometric verification.

Findings

Improved verification accuracy on NIST SRE19 dataset

Effective quality estimation for both modalities

Enhanced fusion performance

Abstract

Fusion of scores is a cornerstone of multimodal biometric systems composed of independent unimodal parts. In this work, we focus on quality-dependent fusion for speaker-face verification. To this end, we propose a universal model which can be trained for automatic quality assessment of both face and speaker modalities. This model estimates the quality of representations produced by unimodal systems which are then used to enhance the score-level fusion of speaker and face verification modules. We demonstrate the improvements brought by this quality-dependent fusion on the recent NIST SRE19 Audio-Visual Challenge dataset.