# Signal Detection Method for OTFS System Based on Adaptive Wavelet Convolutional Neural Network

## TL;DR

This paper introduces a new signal detection method for OTFS systems using an adaptive wavelet convolutional neural network to improve performance and efficiency.

## Contribution

The novel approach replaces fixed convolution kernels with adaptive wavelet layers to better match OTFS signal characteristics.

## Key findings

- The AWCNN model achieves faster convergence and better bit error rate performance at low signal-to-noise ratios.
- Replacing the first CNN layer with an adaptive wavelet layer enhances sparse feature extraction from OTFS signals.
- Incorporating message-passing algorithm estimates improves detection performance.

## Abstract

In Orthogonal Time–Frequency Space (OTFS) systems, signal detection algorithms based on convolutional neural networks (CNNs) suffer from insufficient feature extraction and are limited by local mixing. Additionally, fixed convolution kernels struggle to match the sparsity and non-stationary characteristics of OTFS signals in the delay-Doppler domain, resulting in slow convergence and high training costs. We do not stop at simply integrating more features outside the existing CNN framework. Instead, we go deeper into the network and replace the fixed convolution kernels with wavelet convolution layers that have time–frequency-adaptive capabilities. This fundamental change allows the network to more intrinsically match the physical characteristics of OTFS signals in the delay-Doppler domain, thereby achieving excellent detection performance while also gaining faster convergence efficiency. Therefore, this paper proposes a signal detection method using an adaptive wavelet convolutional neural network (AWCNN). The approach replaces the first convolutional layer of a standard CNN with an adaptive wavelet layer, which leverages the time–frequency localization properties of Sym4 wavelet kernels along with learnable scaling and translation factors. This enhances the network’s ability to extract sparse features from OTFS signals. Additionally, the model incorporates both the original received signal and preliminary estimates from the message-passing (MP) algorithm as input features, enriching the dataset and further improving detection performance. Experimental results demonstrate that the AWCNN model achieves superior convergence efficiency compared to the CNN model, which attains a bit error rate (BER) comparable to that of the CNN algorithm at a low signal-to-noise ratio of 2 dB, operating without the need for pilot-assisted channel state information acquisition.