Blind Equalization using a Variational Autoencoder with Second Order Volterra Channel Model

TL;DR

This paper introduces a novel blind non-linear equalization method using a variational autoencoder with a second-order Volterra channel model, improving performance in complex communication links without requiring labeled data.

Contribution

It develops an analytical ELBO for real-valued constellations in VAE-based blind equalization with a second-order Volterra model, enabling effective unsupervised non-linear equalization.

Findings

Significant performance improvements over linear models

Effective in synthetic Wiener-Hammerstein channels

Applicable to optical communication links

Abstract

Existing communication hardware is being exerted to its limits to accommodate for the ever increasing internet usage globally. This leads to non-linear distortion in the communication link that requires non-linear equalization techniques to operate the link at a reasonable bit error rate. This paper addresses the challenge of blind non-linear equalization using a variational autoencoder (VAE) with a second-order Volterra channel model. The VAE framework's costfunction, the evidence lower bound (ELBO), is derived for real-valued constellations and can be evaluated analytically without resorting to sampling techniques. We demonstrate the effectiveness of our approach through simulations on a synthetic Wiener-Hammerstein channel and a simulated intensity modulated direct detection (IM/DD) optical link. The results show significant improvements in equalization performance, compared to a VAE with linear channel assumptions, highlighting the importance of appropriate channel modeling in unsupervised VAE equalizer frameworks.