Estimating the confidence of speech spoofing countermeasure

TL;DR

This paper explores methods to estimate the confidence of speech spoofing countermeasures, enabling better handling of unknown attacks by deferring decisions when confidence is low.

Contribution

It introduces and evaluates simple confidence estimators that can be integrated into existing speech spoofing countermeasures to improve detection of unknown attacks.

Findings

Energy-based estimator performs well on unknown attacks.

Neural-network-based estimator shows acceptable performance.

Confidence scores help distinguish bona fide and spoofed trials.

Abstract

Conventional speech spoofing countermeasures (CMs) are designed to make a binary decision on an input trial. However, a CM trained on a closed-set database is theoretically not guaranteed to perform well on unknown spoofing attacks. In some scenarios, an alternative strategy is to let the CM defer a decision when it is not confident. The question is then how to estimate a CM's confidence regarding an input trial. We investigated a few confidence estimators that can be easily plugged into a CM. On the ASVspoof2019 logical access database, the results demonstrate that an energy-based estimator and a neural-network-based one achieved acceptable performance in identifying unknown attacks in the test set. On a test set with additional unknown attacks and bona fide trials from other databases, the confidence estimators performed moderately well, and the CMs better discriminated bona fide and spoofed trials that had a high confidence score. Additional results also revealed the difficulty in enhancing a confidence estimator by adding unknown attacks to the training set.