Universal Discrete Filtering with Lookahead or Delay

TL;DR

This paper introduces universal discrete filtering schemes based on sequential probability assignments, capable of estimating source components from noisy outputs with limited lookahead or delay, suitable for resource-constrained scenarios.

Contribution

It proposes a family of universal filtering schemes induced by SPAs, including practical implementations using LZ78, and derives new bounds on Bayesian filtering performance.

Findings

LZ78-based schemes are practically implementable.

Derived new bounds on filtering performance in Bayesian setting.

Schemes perform well with limited lookahead or delay.

Abstract

We consider the universal discrete filtering problem, where an input sequence generated by an unknown source passes through a discrete memoryless channel, and the goal is to estimate its components based on the output sequence with limited lookahead or delay. We propose and establish the universality of a family of schemes for this setting. These schemes are induced by universal Sequential Probability Assignments (SPAs), and inherit their computational properties. We show that the schemes induced by LZ78 are practically implementable and well-suited for scenarios with limited computational resources and latency constraints. In passing, we use some of the intermediate results to obtain upper and lower bounds that appear to be new, in the purely Bayesian setting, on the optimal filtering performance in terms, respectively, of the mutual information between the noise-free and noisy sequence, and the entropy of the noise-free sequence causally conditioned on the noisy one.