Iterative Channel Estimation for Discrete Denoising under Channel Uncertainty

TL;DR

This paper introduces ICE-N-DUDE, an iterative neural network-based algorithm that estimates unknown channels for discrete denoising, demonstrating universal robustness and superior performance over traditional methods across diverse data types.

Contribution

The paper presents a novel iterative channel estimation algorithm that removes the need for known noisy channels, enhancing universal discrete denoising capabilities.

Findings

ICE-N-DUDE performs well regardless of channel and source uncertainties.

The method is highly robust to hyperparameters.

It outperforms the Baum-Welch algorithm in experiments.

Abstract

We propose a novel iterative channel estimation (ICE) algorithm that essentially removes the critical known noisy channel assumption for universal discrete denoising problem. Our algorithm is based on Neural DUDE (N-DUDE), a recently proposed neural network-based discrete denoiser, and it estimates the channel transition matrix as well as the neural network parameters in an alternating manner until convergence. While we do not make any probabilistic assumption on the underlying clean data, our ICE resembles Expectation-Maximization (EM) with variational approximation, and it takes advantage of the property of N-DUDE being locally robust around the true channel. With extensive experiments on several radically different types of data, we show that the ICE equipped N-DUDE (dubbed as ICE-N-DUDE) can perform \emph{universally} well regardless of the uncertainties in both the channel and the clean source. Moreover, we show ICE-N-DUDE becomes extremely robust to its hyperparameters and significantly outperforms the strong baseline that can deal with the channel uncertainties for denoising, the widely used Baum-Welch (BW) algorithm for hidden Markov models (HMM).