Hybrid Iterative Detection for OTFS: Interplay between Local L-MMSE and Global Message Passing

TL;DR

This paper introduces a hybrid iterative detection method for OTFS that combines local L-MMSE estimation with global message passing, improving detection accuracy and reducing complexity in high-mobility channels.

Contribution

A novel hybrid detection framework for OTFS that leverages a delay-Doppler commutation precoder to structure the channel matrix for efficient iterative detection.

Findings

Achieves fast and reliable convergence in detection.

Significantly improves BER performance under various channel conditions.

Near-optimal error performance close to the matched filter bound.

Abstract

Orthogonal time frequency space (OTFS) modulation has emerged as a robust solution for high-mobility wireless communications. However, conventional detection algorithms, such as linear equalizers and message passing (MP) methods, either suffer from noise enhancement or fail under complex doubly-selective channels, especially in the presence of fractional delay and Doppler shifts. In this paper, we propose a hybrid low-complexity iterative detection framework that combines linear minimum mean square error (L-MMSE) estimation with MP-based probabilistic inference. The key idea is to apply a new delay-Doppler (DD) commutation precoder (DDCP) to the DD domain signal vector, such that the resulting effective channel matrix exhibits a structured form with several locally dense blocks that are sparsely inter-connected. This precoding structure enables a hybrid iterative detection strategy, where a low-dimensional L-MMSE estimation is applied to the dense blocks, while MP is utilized to exploit the sparse inter-block connections. Furthermore, we provide a detailed complexity analysis, which shows that the proposed scheme incurs lower computational cost compared to the full-size L-MMSE detection. The simulation results of convergence performance confirm that the proposed hybrid MP detection achieves fast and reliable convergence with controlled complexity. In terms of error performance, simulation results demonstrate that our scheme achieves significantly better bit error rate (BER) under various channel conditions. Particularly in multipath scenarios, the BER performance of the proposed method closely approaches the matched filter bound (MFB), indicating its near-optimal error performance.