Optimal Channel Shortener Design for Reduced-State Soft-Output Viterbi Equalizer in Single-Carrier Systems

TL;DR

This paper proposes an information-theoretic approach to designing a channel shortener for reduced-state Viterbi equalizers in single-carrier systems, improving detection performance at higher code-rates.

Contribution

It introduces a mutual information lower bound-based design for the channel shortener, optimizing the target response with a gradient method and considering decision-feedback quality.

Findings

Achieves superior detection performance at medium and high code-rates.

Provides closed-form solutions for prefilter and feedback filter design.

Analyzes the information-theoretic properties of the proposed design.

Abstract

We consider optimal channel shortener design for reduced-state soft-output Viterbi equalizer (RS-SOVE) in single-carrier (SC) systems. To use RS-SOVE, three receiver filters need to be designed: a prefilter, a target response and a feedback filter. The collection of these three filters are commonly referred to as the \lq\lq{}channel shortener\rq\rq{}. Conventionally, the channel shortener is designed to transform an intersymbol interference (ISI) channel into an equivalent minimum-phase equivalent form. In this paper, we design the channel shortener to maximize a mutual information lower bound (MILB) based on a mismatched detection model. By taking the decision-feedback quality in the RS-SOVE into consideration, the prefilter and feedback filter are found in closed forms, while the target response is optimized via a gradient-ascending approach with the gradient explicitly derived. The information theoretical properties of the proposed channel shortener are analyzed. Moreover, we show through numerical results that, the proposed channel shortener design achieves superior detection performance compared to previous channel shortener designs at medium and high code-rates.