Supervised Machine Learning for Signals Having RRC Shaped Pulses

TL;DR

This paper compares supervised machine learning methods like SVMs, neural networks, and logistic regression for modulation recognition of signals with RRC-shaped pulses under extreme noisy conditions, using simple statistical features.

Contribution

It introduces a robust feature set based on sample mean and variance for distinguishing FSK and QAM-PSK signals in challenging environments.

Findings

Support vector machines outperform other classifiers in noisy conditions.

Simple statistical features effectively differentiate modulation types.

Robust classification achieved despite joint impairments and noise.

Abstract

Classification performances of the supervised machine learning techniques such as support vector machines, neural networks and logistic regression are compared for modulation recognition purposes. The simple and robust features are used to distinguish continuous-phase FSK from QAM-PSK signals. Signals having root-raised-cosine shaped pulses are simulated in extreme noisy conditions having joint impurities of block fading, lack of symbol and sampling synchronization, carrier offset, and additive white Gaussian noise. The features are based on sample mean and sample variance of the imaginary part of the product of two consecutive complex signal values.

Equations

$$s(t)=x(t-t_0)e^{j(\Delta t+{\theta }_c)}\alpha (t)e^{j\psi (t)}+v(t)$$

$$x(t-t_0)=\sum _{n=-\infty }^{+\infty }{a}_n{e}^{j{\theta }_n}p(t-t_0-nT)$$

$$x(t-t_0)=e^{j\underset{0}{\overset{t-t_0}{\int }}\sum _{n=-\infty }^{+\infty }{b}_{n}q(\rho -nT)d\rho }$$

$$T=N_s{T}_s+\epsilon T_s$$

$$s[k]=x(k{T}_s-t_0)e^{j({\Delta }^{\prime }k+{\theta }_c)}\alpha [k]e^{j\psi [k]}+v[k]=s(t){|}_{t=k{T}_s}$$

$$w[k]=s[k]s^{\ast }[k-1]$$

$$\begin{array}{rl}w[k]{|}_{v[k]\equiv 0}& =\sum _{n=-\infty }^{+\infty }{a}_n^2{e}^{j{\Delta }^{\prime }}{P}_n[k]P_n^{-}[k]+\\ & \sum _{n=-\infty }^{+\infty }\sum _{m=-\infty }^{+\infty }{a}_n{a}_m{e}^{j[{\theta }_n-{\theta }_m+{\Delta }^{\prime }]}{P}_n[k]P_m^{-}[k]\end{array}$$

$$\begin{array}{rl}w[k]{|}_{v[k]\equiv 0}& =e^{j[\underset{k{T}_s-T_s-t_0}{\overset{k{T}_s-t_0}{\int }}{\omega }^{\prime }(\rho )\frac{d\rho }{T_s}+{\Delta }^{\prime }]}.\end{array}$$

$$\begin{array}{rl}E[\text{mbox}Im(w[k]{|}_{v[k]\equiv 0})]=& E[\text{mbox}Im(w[k])]\\ & =\text{mbox}sin({\Delta }^{\prime })\sum _{n=-\infty }^{+\infty }{P}_n[k]P_n^{-}[k]E[a_n^2]\end{array}$$

$$E[\text{mbox}Im(w[k]{|}_{v[k]\equiv 0})]=E[\text{mbox}Im(w[k])]=\sin ({\Delta }^{\prime })\prod _{m=-\infty }^{\infty }\cos (Q_m[k])$$

$$Q_m[k]=\underset{k{T}_s-T_s-t_0}{\overset{k{T}_s-t_0}{\int }}q(p-mT)d\rho .$$

$$\begin{split}\mbox{VAR}[\mbox{Im}(w[k]|_{v[k]\equiv 0})]&=\sin^{2}(\Delta^{\prime})\big{[}(\sum\limits_{m=-\infty}^{+\infty}P_{m}[k]P_{m}^{-}[k])^{2}+\\ &\sum\limits_{m=-\infty}^{+\infty}P_{m}^{2}[k]\sum\limits_{m=-\infty}^{+\infty}(P_{m}^{-}[k])^{2}-2\sum\limits_{m=-\infty}^{+\infty}(P_{m}[k]P_{m}^{-}[k])^{2}\big{]}.\end{split}$$

$$\begin{split}\mbox{VAR}[\mbox{Im}(w[k]|_{v[k]\equiv 0})]&=\sin^{2}(\Delta^{\prime})\bigg{[}(E[a_{0}^{4}]-2E^{2}[a_{0}^{2}])\\ &\sum\limits_{m=-\infty}^{+\infty}(P_{m}[k]P_{m}^{-}[k])^{2}+E^{2}[a_{0}^{2}](\sum\limits_{m=-\infty}^{+\infty}P_{m}[k]P_{m}^{-}[k])^{2}\bigg{]}+\frac{1}{2}E^{2}[a_{0}^{2}]\times\\ &\bigg{[}\sum\limits_{m=-\infty}^{+\infty}P_{m}^{2}[k]\sum\limits_{m=-\infty}^{+\infty}(P_{m}^{-}[k])^{2}-(\sum\limits_{m=-\infty}^{+\infty}P_{m}[k]P_{m}^{-}[k])^{2}\bigg{]}.\end{split}$$

$$\begin{split}\mbox{VAR}[\mbox{Im}(w[k]|_{v[k]\equiv 0})]&=\frac{1}{2}-\frac{1}{2}\prod\limits_{m=-\infty}^{+\infty}\cos(2Q_{m}[k])+\\ &\sin^{2}(\Delta^{\prime})\big{(}\prod\limits_{m=-\infty}^{+\infty}\cos(2Q_{m}[k])-\prod\limits_{m=-\infty}^{+\infty}\cos^{2}(Q_{m}[k])\big{)}.\end{split}$$

$$L(\theta)=\sum_{i=1}^{m}\big{(}y_{i}\mbox{log}(h(x_{i}))+(1-y_{i})\mbox{log}(1-h(x_{i}))\big{)}$$

$$h(x_i)=\frac{1}{1+e^{-{\theta }_0-{\sum }_{j=1}^u{\theta }_{ij}{x}_{ij}}}$$

$$h=2f_{d}T$$

$$h=(N_s+ϵ)\delta /\pi$$

Full text

I Introduction

Signal separation (SS) is used in applications such as interference identification, electronic warfare, enforcement of civilian spectrum compliance, radar, intelligent modems, cognitive and software defined radios. Most of the published work involving the above mentioned communication systems assume that the signals and their parameters are known. For instance , the works in [1] and [2] do not focus on the SS part. Understandably the focus is on the application itself. This paper studies the SS in noisy conditions that serve as a prerequisite for several applications including but not limited to those mentioned above.

The features in this work, discussed for several scenarios in [3]-[7], separate the continuous phase frequency shift keying (CPFSK) modulation from linear modulations of phase shift keying (PSK) and quadrature amplitude modulation (QAM). Root raised cosine (RRC) shaped pulses are used to generate the signals. Modulations are simulated to have the joint presence of block fading, lack of symbol and sampling synchronization, carrier offset, additive white Gaussian noise (AWGN) and unknown pulse shape (unknown roll-off). None of the mentioned works discuss the effects of block fading on the performance of SS. Also, support vector machines (SVM) are the only pattern recognition-based approach being employed in the above mentioned works. In this paper, block fading is introduced to make the conditions more challenging. Furthermore, this work compares the classification performances of the SVM, logistic regression (LR) and neural networks (NN) for the same set of features. The order of modulation of the CPFSK signals (binary, 4-ary and 8-ary CPFSK) is identified in [8]. In [9], frequency modulated signals are separated from the linearly modulated signals.

II Signal Model

We model the complex baseband continuous-time received signal as [4]

[TABLE]

where $x(t)$ is the transmitted-signal, $\theta_{c}$ is the initial phase uniformly distributed over $[0,~{}2\pi)$, $\Delta$ is the carrier offset, $\alpha(t)e^{j\psi(t)}$ models the fading. $\alpha(t)$ is a Rayleigh random variable. $\psi(t)$ is uniformly distributed over $[0,2\pi)$ and is independent of $\alpha(t)$. $v(t)$ is zero-mean complex noise and $t_{0}$ is the time delay. Furthermore, it is assumed that the bandwidth $B$ of the receiver’s filter is in general larger than the bandwidth of the transmitted signal and the power spectral density of $v(t)$ does not vary within the filter’s bandwidth. In this work the center of the signal’s spectrum is translated to be around a certain desired normalized frequency, say [math] or $\pi/2$.

For linear modulations the transmitted-signal is

[TABLE]

where $(a_{n},~{}\theta_{n})$ are the amplitude and phase of the transmitted symbol, $p(t)$ is the pulse shape function and $T$ is the symbol period. In the case of FSK modulation

[TABLE]

where $q(\rho)$ defines instantaneous frequency pulse shape and $b_{n}\in\{-1,+1\}$ for BFSK. Note that (3) models continuous phase FSK, which is of higher practical interest than non-continuous phase FSK used in [10].

Sampling period, $T_{s}$, is assumed to be $1/(2B)$. Symbol period is given by

[TABLE]

where $N_{s}$ is the oversampling and $\varepsilon$ is uniformly distributed in $[0,1)$. Finally the complex baseband discrete-time received signal is

[TABLE]

where $\Delta^{\prime}=\Delta T_{s}$, $\alpha[k]=\alpha(kT_{s})$, $\psi[k]=\psi(kT_{s})$ and $v[k]$ is complex circular AWGN having zero mean and unit variance. Note that $t_{0}=k_{0}T_{s}+\varepsilon_{0}T_{s}$ where $k_{0}$ is the integer part and $\varepsilon_{0}\in[0,1)$ is the fractional part of the time delay $t_{0}$ measured in sampling periods as time units.

III The Three Features

Let

[TABLE]

where * is the complex conjugate operator. We assume that $\alpha[k]=1,~{}\psi[k]=0$, i.e., fading is skipped for the sake of simplicity. $w[k]$ for noiseless QAM-PSK signals is

[TABLE]

where $P_{n}[k]P_{m}^{-}[k]=p(kT_{s}-t_{0}-nT)p(kT_{s}-T_{s}-t_{0}-mT).$ Similarly, applying $w[k]$ on noiseless FSK signals yields

[TABLE]

Let us consider the means of imaginary part of $w[k]$ in (7) and (8) for linear and BFSK modulations, respectively. For equiprobable constellation points of each modulation, mean of imaginary part of $w[k]$ for 16-QAM, BPSK, 4-PSK and 8-PSK signals is given by

[TABLE]

where 16-QAM has constellation points $a_{n}e^{j\theta_{n}}\in\{k/\sqrt{10}+jl/\sqrt{10};~{}k,~{}l=-3,-1,+1,+3\}$, BPSK’s phases $\theta_{n}\in\{0,~{}\pi\}$, 4-PSK’s phases $\theta_{n}\in\{(2n+1)\pi/4;~{}n=0,1,2,3\}$, and 8-PSK’s phases $\theta_{n}\in\{(2n+1)\pi/8;~{}n=0,1,...,7\}$. Constellation points for 16-QAM, BPSK, 4-PSK and 8-PSK are chosen such that the average power is unity. Mean of imaginary part of $w[k]$ in (8) for BFSK signal is

[TABLE]

where

[TABLE]

Next, let us consider the variances of imaginary part of $w[k]$ in (7) for QAM and PSK modulations, and in (8) for BFSK modulation. The variance for BPSK is

[TABLE]

For QAM, 4-PSK and 8-PSK modulations, whose signal constellations are invariant to $\pi/2$ rotation, the variance is

[TABLE]

The variance of $\mbox{Im}(w[k])$ in (8) for BFSK modulation is

[TABLE]

IV Supervised Learning Techniques

In this work, classification performances of the SVM, LR and NN are compared for the features introduced in [3]. The features are

sample mean of $\mbox{Im}(w[k])$ for the signal $s[k]$ obtained by frequency downconversion to $\pi/2$, 2. 2.

sample variance of $\mbox{Im}(w[k])$ for the signal $s[k]$ obtained by frequency downconversion to 0, and 3. 3.

sample variance of $\mbox{Im}(w[k])$ for the signal $s[k]$ obtained by frequency downconversion to $\pi/2$.

The features based on $\mbox{Im}(w[k])$ for the signal $s[k]$ obtained by frequency downconversion to $\pi/2$ can be seen as based on $\mbox{Re}(w[k])$ for the signal $s[k]$ obtained by frequency downconversion to 0.

IV-A SVM

SVM are initially introduced in [11]. The current soft margin form of SVM is presented in [12]. As in any pattern recognition-based approach, SVM create a classification model for the features of the known set of training realizations. The classification model can be linear as well as non-linear. This work uses the linear kernel because the number of features is small and a simple linear decision boundary suffices to separate the signals. For higher dimensional feature space, a more complicated decision boundary, a non-linear one, should be used to avoid the underfitting. It is desirable to have the separation between the features of two classes as wide as possible. The features of the new realizations are then predicted to belong to the either class depending upon which side of the decision boundary they lie. A detailed tutorial can be found in [13].

IV-B LR

For $m$ training examples, LR is the problem of maximizing the following log-likelihood

[TABLE]

where $x_{i}$ is the $i^{\mbox{th}}$ training feature vector example. In this work, $x_{i}$ is a 3-dimensional feature vector $[x_{i1},x_{i2},x_{i3}]^{T}$, $y_{i}$ is the $i^{\mbox{th}}$ target value. $y_{i}\in\{0,1\}$ and

[TABLE]

where $u$ is dimension of the feature vector $x_{i}$.

IV-C NN

In this work, the input layer has three nodes, that is, the dimension of the feature vector is 3. There is only one hidden layer having 10 nodes. Since this work discusses the binary classification problem, the output layer has 1 node only.

V Simulations and Discussion

The simulation experiments are presented illustrating the classification performance of the SVM, LR and NN for the signals having RRC shaped pulses. The performance is measured by the accuracy, which is total number of correct classifications divided by total number of signals.

For CPFSK signals, the modulation index is defined as

[TABLE]

where $f_{d}$ is the maximum frequency deviation. Using (4) and (17) we get

[TABLE]

where $\delta=2\pi f_{d}T_{s}$. For BFSK $\delta=\beta^{\prime}$ where $\beta^{\prime}/T_{s}$ is the amplitude of the instantaneous frequency pulse $q(t)$ in (LABEL:e3a). In this work, FSK signals have carrier frequencies $\beta_{mi}^{\prime}\in\{\Delta^{\prime}+(2i-(m+1))h\pi/((m-1)(N_{s}+\epsilon));~{}m=2,4,8;i=1,2,....,m\}$, where $\Delta^{\prime}$ is uniformly distributed in $[\gamma^{\prime}-\pi/20,~{}\gamma^{\prime}+\pi/20]$. The value of $h$ is inversely proportional to the channel’s spectral efficiency. Therefore CPFSK signals having $h<1$ are more useful than those where $h\geq 1$. Also, SS for $h<1$ is the more challenging scenario than that of $h\geq 1.$ Because of the presence of block fading, the SS performance for different values of $h<1$ are very similar. Therefore, only the results for $h=1/2$ are presented in this work.

There are 10000 signals for each modulation (BFSK, 4-FSK, 8-FSK, BPSK, 4-PSK, 8-PSK and 16-QAM). Therefore the total number of modulated signals is 70000. There are 600 symbols in one realization and the oversampling factor $N_{s}=6$. Carrier offset, $\Delta^{\prime}$, is uniformly distributed in $[\gamma^{\prime}-\pi/20$, $\gamma^{\prime}+\pi/20]$, where $\gamma^{\prime}\in\{0,~{}\pi/2\}.$ Roll-off of the RRC pulses is in $\{k/10;~{}k=1,~{}2,~{}3,....,~{}10\}$. Both $\Delta^{\prime}$ and roll-off of the RRC pulses are fixed for a realization and they vary independently from realization to realization following uniform distributions. $\varepsilon$ and $\varepsilon_{0}$ are both uniformly distributed in [0,1), which means that $T$ and time delay are non-integer multiples of sampling period. This results in asynchronicity between sampling instants and symbol period. $\varepsilon$ and $\varepsilon_{0}$ remain unchanged for a particular realization. $k_{0}$ is uniformly distributed in $\{0,1,2,...,\lceil N_{s}+\epsilon\rceil-1\}$. For block fading, $\alpha[k]e^{i\psi[k]}$ is a constant for a realization and varies independently for each realization. $\alpha[k]$ has unit mean square value, that is, $E[\alpha^{2}[k]]=1$ and $\psi[k]$ is uniformly distributed in $[0,2\pi)$.

The signals are simulated to have the joint presence of block fading, unknown roll-off, lack of symbol and sampling synchronization, carrier offset and AWGN.

Figure 1 represents the accuracies of the SVM, LR and NN. For each classifier, the number of training realizations is 7000 (1000 realizations for each modulation) for a particular value of SNR. For this large number of training realizations, NN performs the best, especially for smaller values of SNR. LR performs marginally better than the SVM.

Figures 2, 3 and 4 represent the performances of the SVM, LR and NN, respectively. The performances are shown for different number of training realizations. The performance of the particular classifier do not seem to improve when the number of training realizations is increased from 1000 to 2000 for each modulation. Compared to NN, both SVM and LR perform consistently better with the increasing SNRs for the small number of training (50). The accuracies of the SVM and LR for 50 realizations are comparable to those of 1000 and 2000 training realizations. NN, on the other hand, performs poorly for 50 realizations as compared to its performances for 1000 and 2000 training realizations.

Figure 5 represents the accuracies for a range of training realizations for a fixed value of SNR = 5 dB. It can be seen that the NN performs the best for a larger number of training realizations and LR performs marginally better than the SVM.

VI Conclusions

The features are classified by SVM, LR and NN, trained for different number of realizations. Each classifier performs similarly for training of 1000 and 2000 realizations. For a low number of 50 training realizations, NN performs poorly. Both SVM and LR perform significantly better than the NN. For a larger number of 1000 or 2000 training realizations, NN performs slightly better than the others for SNRs less than 10 dB. The performances become identical after SNR of 10 dB. If more training realizations are available then NN should be used because of its superior performance for lower values of SNR. If fewer training realizations are available then because of the slightly better performance, especially for lower values of SNR, LR should be preferred over SVM.