DNN Filter Bank Cepstral Coefficients for Spoofing Detection

TL;DR

This paper introduces a new deep neural network filter bank cepstral feature (DNN-FBCC) for detecting spoofed speech, improving robustness against unknown attacks in speaker verification systems.

Contribution

It proposes a novel learned filter bank feature using a neural network trained with restrictions, capturing speech differences more effectively than traditional methods.

Findings

DNN-FBCC outperforms LFCC in detection accuracy.

The method is especially effective against unknown spoofing attacks.

Experimental results on ASVspoof 2015 demonstrate improved reliability.

Abstract

With the development of speech synthesis techniques, automatic speaker verification systems face the serious challenge of spoofing attack. In order to improve the reliability of speaker verification systems, we develop a new filter bank based cepstral feature, deep neural network filter bank cepstral coefficients (DNN-FBCC), to distinguish between natural and spoofed speech. The deep neural network filter bank is automatically generated by training a filter bank neural network (FBNN) using natural and synthetic speech. By adding restrictions on the training rules, the learned weight matrix of FBNN is band-limited and sorted by frequency, similar to the normal filter bank. Unlike the manually designed filter bank, the learned filter bank has different filter shapes in different channels, which can capture the differences between natural and synthetic speech more effectively. The experimental results on the ASVspoof {2015} database show that the Gaussian mixture model maximum-likelihood (GMM-ML) classifier trained by the new feature performs better than the state-of-the-art linear frequency cepstral coefficients (LFCC) based classifier, especially on detecting unknown attacks.

Equations

$$\mathbf{H}1=\mathbf{F}{\mathbf{W}}_{\mathbf{fb}},$$

$${\mathbf{W}}_{\mathbf{fb}}=\mathrm{NR}(\mathbf{W})\odot {\mathbf{M}}_{\mathbf{bl}},$$

$$\mathrm{NR}(x)=1/(1+exp(-x)).$$

$$W_{dc}=W_{dc}-\eta g_{new},$$

$$g_{new}=(1-m)\times g+m\times g_{old},$$

$$g=\frac{\partial L}{\partial {H1}_c}\frac{\partial {H1}_c}{\partial {\mathrm{W}}_{dc}}=\frac{\partial L}{\partial {H1}_c}{F}_d{M}_{b{l}_{dc}}\frac{\partial \mathrm{NR}(W_{dc})}{\partial W_{dc}},$$

$$\frac{\partial \mathrm{NR}(W_{dc})}{\partial W_{dc}}=\mathrm{NR}(W_{dc})[1-\mathrm{NR}(W_{dc})],$$

$$\mathbf{ML}\left(\text{X}\right)=\frac{1}{T}\sum _{i=1}^T\left\{\log \mathrm{P}({\mathbf{X}}_i|{\lambda }_{\mathrm{human}})-\log \mathrm{P}({\mathbf{X}}_i|{\lambda }_{\mathrm{spoof}})\right\},$$

Full text

I Introduction

As a low-cost and flexible biometric solution to person authentication, automatic speaker verification (ASV) has been used in many telephone or network access control systems, such as telephone banking [1]. Recently, with the improvement of automatic speech generation methods, speech produced by voice conversion (VC) [2][3] and speech synthesis (SS) [4][5] techniques has been used to attack ASV systems. Over the past few years, much research has been devoted to protect ASV systems against spoofing attack [6][7][8].

There are two general strategies to protect ASV systems. One is to develop a more robust ASV system which can resist the spoofing attack. Unfortunately, research has shown that all the existing ASV systems are vulnerable to spoofing attacks [9][10][11]. Verification and anti-spoofing tasks can not be done well in only one system at the same time.

The other more popular strategy is to build a separated spoofing detection system which only focuses on distinguishing between natural and synthetic speech [12]. Because of the advantage of being easily incorporated into existing ASV systems, spoofing detection has become an important research topic in anti-spoofing [6][10][13][14].

Many different acoustic features have been proposed to improve the performance of Gaussian mixture model maximum-likelihood (GMM-ML) based spoofing detection systems. In [15], relative phase shift (RPS) and Mel-frequency cepstral coefficients (MFCC) were used to detect SS attacks. A fusion system combining MFCC and group delay cepstral coefficients (GDCC) was applied to resist VC spoofing in [1]. Paper [16] compared the spoofing detection performance of 11 different features on the ASVspoof 2015 database [17]. Among others, dynamic linear frequency cepstral coefficients (LFCC) feature performed best on the evaluation set and the average equal error rate was lower than 1%.

Different from the aforementioned systems, some more general systems using machine learning methods were developed to model the difference between natural and synthetic speech more effectively. In [18][19][20], spoofing detection systems based on deep neural networks (DNNs) were proposed and tested, where a DNN was used as a classifier or feature extractor. Unfortunately, experimental results showed that, compared with the acoustic feature based GMM-ML systems, these DNN systems performed slightly better on detecting the trained/known spoofing methods, but much worse on detecting unknown attacks.

In the previous studies, when a DNN was used as a feature extractor, the output of the middle hidden layer was used as DNN features to directly train some other types of models, e.g., Gaussian mixture model (GMM) or support vector machine (SVM) [19][21][22].

If we use the short-term power spectrum as the input of a DNN and set the activation function of first hidden layer as “linear”, the learned weight matrix between the input layer and the first hidden layer can be considered as a special type of learned filter bank. The number of this hidden layer nodes corresponds to the number of filter bank channels and each column of the weigh matrix can be treated as the frequency response of each filter. Unlike the conventional manually designed filter banks, the filters of the learned filter bank have different shapes in different channels, which can capture the discriminative characteristic between natural and synthetic speech more effectively. The DNN feature generated from the first hidden layer can be treated as a kind of filter bank feature.

Some filter bank learning methods such as LDA (Linear discriminant analysis) filter learning [23] and log Mel-scale filters learning [24] have been introduced in the literatures. These methods did not restrict the shapes of learned filters and the learned filter bank features were used on the speech recognition task.

In this paper, we introduce a new filter bank neural network (FBNN) by introducing some restriction on the training rules, the learned filters are non-negative, band-limited, ordered by frequencies and have restricted shapes. The DNN feature generated by the first hidden layer of FBNN has the similar physical meaning of the conventional filter bank feature and after cepstral analysis we obtain a new type of feature, namely, deep neural network filter bank cepstral coefficients (DNN-FBCC). Experimental results show that the GMM-ML classifier based on DNN-FBCC feature outperforms the LFCC feature and DNN feature on the ASVspoof 2015 data base [17].

II Filter Bank Neural Networks

As a hot research area, deep neural networks have been successfully used in many speech processing tasks such as speech recognition [25][26], speaker verification [27][28] and speech enhancement [29][30].

A trained DNN can be used for regression analysis, classification, or feature extraction. When a DNN is used as a feature extractor, due to lack of knowledge about the specific physical interpretation of the DNN feature, the learned feature can only be used to train some other models, directly. Further processing, such as cepstral analysis, can not be applied.

As one of the most classical features for speech processing, cepstral (Cep) features , e.g., MFCC and LFCC, have been widely used in most speech processing tasks.

Cep features can be created with the following procedure shown in Fig.1. Firstly, the speech signal is segmented into short time frames with overlapped windows. Secondly, the power spectrum $\left|\mathbf{X}\left(e^{jw}\right)\right|^{2}$ are generated by frame-wise $N$ points fast Fourier transform (FFT). Thirdly, the power spectrum is integrated using overlapping band-limited filter bank with $C$ channels, generating the filter bank features. Finally, after logarithmic compression and discrete cosine transform (DCT) on the filter bank feature, $M$ coefficients are selected as the Cep feature.

As shown in Fig.2.(a), a representative of commonly filters bank used in Cep feature extraction are non-negative, band limited, sorted by frequency and have similar shapes in different channels. The similar shapes for all the channels are not suitable for the spoofing detection task because different frequency bands may play different roles in spoofing attacks. This motivates us to use a DNN model to train a more flexible and effective filter bank.

As show in Fig. 3 we build a FBNN which includes a linear hidden layer H1, a sigmoid hidden layer H2 and a softmax output layer. The number of nodes in the output layer is $N_{out}$, where the first node stands for the human voice and the other nodes represent different spoofing attack methods. The same as computing Cep features, we also use the power spectrum as the input. Because the neural activation function of H1 is a linear function, the output of the first hidden layer can be defined as:

[TABLE]

where $\mathbf{F}$ is the input power spectrum feature with D dimension, $\emph{D}=0.5N+1$. The weight matrix between the input layer and the first hidden layer is defined as a filter bank weight matrix $\mathbf{W_{fb}}$ with dimensions $D\times C$. $C$ is the number of nodes of the first hidden layer and also means the number of channels in the learned filter bank. Each column of $\mathbf{W_{fb}}$ can be treated as a learned filter channel.

If we do not add any restrictions in the training processing, the learned filters will have the shapes as shown in Fig. 2.(b). Each channel can learn a different filter shape but the characteristics of a normal filter bank, such as non-negative, band-limit and ordered by frequency, can not be satisfied.

In order to tackle this problem, we apply some restrictive conditions on $\mathbf{W_{fb}}$ as

[TABLE]

where $\mathbf{W}\in\mathbb{R}^{D\times C}$, $\mathbf{M_{bl}}\in\mathbb{R}_{\geq 0}^{{D\times C}}$ and $\odot$ means element wise multiplication.

$\mathrm{NR}(\cdot)$ is a non-negative restriction function which can make elements of $\mathbf{W_{fb}}$ non-negative. Any monotone increasing function with non-negative output can be used. We select the sigmoid function:

[TABLE]

$\mathbf{M_{bl}}$ is a non-negative band-limiting shape restriction mask matrix which can restrict the filters of the learned filter bank to have limited band, regulation shape and ordered by frequency. $\mathbf{M_{bl}}$ can be generated from any band-limited filter bank by frequency-domain sampling. Fig. 2.(c) shows a $\mathbf{M_{bl}}$ sampling from a linear frequency triangular filter bank with five channels (Fig. 2.(a)).

$W_{dc}$, elements of $\mathbf{W}$, can be learned through stochastic gradient descent using equations (4) - (7):

[TABLE]

[TABLE]

[TABLE]

[TABLE]

where $d\subseteq[1,D]$, $c\subseteq[1,C]$, $\eta$ is the learning rate, $m$ is the momentum, $g$ is the gradient computed in backward pass, $g_{old}$ is the gradient value in the previous mini-batch, and $g_{new}$ is the new gradient for the current min-batch. $L$ is the cost function and $\frac{\partial L}{\partial\mathit{H1}_{c}}$ can be computed by the standard back propagation equations for neural networks [31]. The learned filters with restrictions are illustrated in Fig. 2.(d), which are band limited, ordered by frequency and have different filter shapes in different channels.

Following the cepstral analysis steps we can generate a new kind of Cep features using the filter bank generated from FBNN, which is defined as deep neural networks filter bank cepstral coefficients (DNN-FBCC). The new feature can integrate the advantages of Cep feature and the discrimination ability of DNN model, which are specially suitable for the task of spoofing detection.

III Experimental Results and Discussions

III-A Database and Data Preparation

The performance of spoofing detection using the DNN-FBCC feature is evaluated on the ASVspoof 2015 database [17]. As shown in TABLE I, the database includes three sub datasets without target speaker overlap: the training set, the development set and the evaluation set. We used the training set for FBNN and human/spoof classifier training. The development set and evaluation set were used for testing.

Training set and development set are attacked by the same five spoofing methods, where $S1$, $S2$ and $S5$ belong to VC method and $S3$, $S4$ belong to SS method. Regarding the evaluation set, besides the five known spoofing methods, there are another five unknown methods, where $S6$-$S9$ are VC methods and $S10$ is an SS method.

The speech signals were segmented into frames with $20ms$ length and $10ms$ step size. Pre-emphasis and a hamming window were applied on the frames before the spectrum computation. Paper [16] showed that all the frames of speech are useful for spoofing detection, so we did not apply any voice activity detection method.

III-B FBNN Training

The FBNN described in Section II was built and trained with computational network toolkit (CNTK) [32].

The output layer has five nodes, the first one is for human speech and the other four are for five known spoofing methods (S3 and S4 use the same label). The number of nodes in hidden layer H2 is set as 100, the cross entropy function was selected as the cost function $L$ and the training epoch was chosen as 30. The mini-batch size was set as 128. $\mathbf{W}$ was initialized with uniform random numbers. $\eta$ and $m$ are set as 0.1 and 0 in the first epoch, 1 and 0.9 in the other epochs.

Some experimental results published in paper [33] and [16], show that the high frequency spectrum of speech is more effective for synthetic detection. In order to investigate the affect of different band-limiting and shape restrictions to the learned filter banks, we use four different manually designed filter banks to generate $\mathbf{M_{bl}}$: the linear frequency triangular filter bank (TFB) with 20 channels, the linear frequency rectangular filter bank (RFB) with 20 channels, the equivalent rectangular bandwidth (ERB) space Gammatone filter bank (GFB) with 128 channels, and the inverted ERB space Gammatone filter bank (IGFB) with 128 channels, according to the recommended in paper [34] [16].

Correspondingly, the number of nodes in the first hidden layer were set as 20, 20, 128, 128 for TFB, RFB, GFB and IGFB, respectively. When using TFB and RFB, the dimension of the input power spectrum is 257. The feature dimension is 513 when using GFB and IGFB.

TFB and RFB equally distribute on the whole frequency region (Fig.4 and Fig.4). GFB which has been successfully used in audio recognition [34][35], has denser spacing in the low-frequency region (Fig.4) and IGFB gives higher emphasis to the higher frequency region(Fig.4).

As shown in Fig.4, after training we can get the DNN-triangle filter bank (DNN-TFB), the DNN-rectangle filter bank (DNN-RFB), the DNN-Gammatone filter bank (DNN-GFB) and the DNN-inverted Gammatone filter bank(DNN-IGFB). The learned filters have flexible shapes in different frequency bands which can capture the difference between human and spoofed speech more effectively.

III-C Classifier

In designing the classifier, we train two separated GMMs with 512 mixtures to model natural and spoofed speech, respectively. Log likelihood ratio is used as criterion of assessment, which is defined as:

[TABLE]

where X denotes feature vectors with $T$ frames, $\lambda_{\mathrm{human}}$ and $\lambda_{\mathrm{spoof}}$ are the GMM parameters of human and spoof model, respectively.

III-D Results and Discussions

We compare the spoofing detection performance between four manually designed Cep features and four DNN-FBCC features.

As shown in Table II, manually designed Cep features: LFCC, RFCC (linear frequency rectangle filter bank cepstral coefficients), GFCC (ERB space Gammatone filter bank cepstral coefficients) and IGFCC (inverted ERB space Gammatone filter bank cepstral coefficients) are generated by manually designed filter bank TFB, RFB, GFB and IGFB described in Section III-B. Four DNN-FBCC features, DNN-LFCC, DNN-RFCC, DNN-GFCC and DNN-IGFCC are generated by learned filter banks DNN-TFB, DNN-RFB, DNN-GFB and DNN-IGFB, respectively. The number of coefficients $M$ of all the eight features are set as 20 (including the 0’th coefficient).

Inspired by the work in [16], we use $\Delta$ and $\Delta^{2}$ (first- and second-order frame-to-frame difference) coefficients to train the GMM-ML classifier. Equal error rate (EER) is used for measuring spoofing detection performance. The average EERs of different spoofing methods on development and evaluation set are shown in TABLE III.

We first conduct experiments on four manually designed Cep features, among which, IGFCC($\Delta\Delta^{2}$) performs best on detecting both known and unknown attacks and GFCC($\Delta\Delta^{2}$) works worst. It can be inferred that the filter banks, which give higher emphasis to the higher frequency region, are more suitable for the spoofing detection task. This is inline with the finding in paper [33].

Then we investigate the performance of four DNN-FBCC features. DNN-RFCC($\Delta\Delta^{2}$) performs best on detecting known attacks, but works worse on unknown spoofing attacks. This phenomena shows that the shape restrictions applied on FBNN affect the performance of spoofing detection. When a rectangle filter is selected (RFB, Fig.4), there are no special shape restrictions on the learned filters, and this make the learned DNN-RFCC($\Delta\Delta^{2}$) over-fits the trained/known attacks. When a Gammatone filter is chosen (IGFB, Fig.4), the shape restriction can make the performance of DNN-IGFCC($\Delta\Delta^{2}$) better than the corresponding IGFCC($\Delta\Delta^{2}$) on both known and unknown attacks.

In general, among the eight investigated Cep features, DNN-IGFCC($\Delta\Delta^{2}$), generated by the learned filter bank which has denser spacing in the high frequency region and has the Gammatone shape restriction, performs best on ASVspoof 2015 data base and gets the best average accuracy, overall.

We also compare the DNN-FBCC feature with other three data driven features which have been successfully used in speaker verification and speech recognition task: LDA filter bank feature (LDA-FB) [23], log-normalized learned Mel-scale filter bank feature (l-LMFB) [24] and DNN bottle neck feature (DNN-BN) [21].

LDA-FB is generated by a 20 channels LDA filter bank which is learned by power spectrum feature with 257 dimension.

DNN-BN is produced by the middle hidden layer of a five hidden layers DNN, and the nodes number of hidden layers are set as 2048, 2048, 60, 2048 and 2048, respectively. The DNN is trained by a block of 11 frames of 60 MFCC(static+$\Delta\Delta^{2}$) features.

l-LMFB is generated by a neural network introduced by [24] which uses a 20 channel mel-scale rectangle filter bank to generate $\mathbf{M_{bl}}$ and chooses exponential function $e^{x}$ as a non-negative restriction function.

From the results shown in TABLE III we observe that the simple data driven filter bank feature LDA-FB is not suitable for the spoofing detection task. Static DNN-BN, DNN-BN($\Delta\Delta^{2}$), static l-LMFB and l-LMFB($\Delta\Delta^{2}$) are all perform worse than the DNN-IGFCC($\Delta\Delta^{2}$) feature.

To sum up the learned filter banks produced by FBNN using suitable band limiting and shape restrictions can improve the spoofing detection accuracy over the existing manually designed filter banks by learning flexible and effective filters. DNN-FBCC, especially DNN-IGFCC($\Delta\Delta^{2}$), can largely increase the detection accuracy on unknown spoofing attacks.

IV Conclusions

In this paper, we introduced a filter bank neural network with two hidden layers for spoofing detection. During training, a non-negative restriction function and a band-limiting mask matrix were applied on the weight matrix between the input layer and the first hidden layer. These restrictions made the learned weight matrix non-negative, band-limited, shape restriction and ordered by frequency. The weight matrix can be used as a filter bank for cepstral analysis. Experimental results show that cepstral coefficients (Cep) features produced by the learned filter banks were able to distinguish the natural and synthetic speech more precisely and robustly than the manually designed Cep features and general DNN features.