A Weyl Criterion for Finite-State Dimension and Applications

TL;DR

This paper extends the classical Weyl criterion to characterize finite-state dimensions of sequences, providing a new analytical tool for studying data compression, prediction, and normality in sequences.

Contribution

It generalizes the Weyl criterion from normality (dimension 1) to all finite-state dimensions, linking exponential sums with finite-state dimension.

Findings

Extended Weyl criterion characterizes all finite-state dimensions.

Finite-state dimension relates to subsequence limits of exponential sums.

Demonstrated utility through illustrative examples.

Abstract

Finite-state dimension, introduced early in this century as a finite-state version of classical Hausdorff dimension, is a quantitative measure of the lower asymptotic density of information in an infinite sequence over a finite alphabet, as perceived by finite automata. Finite-state dimension is a robust concept that now has equivalent formulations in terms of finite-state gambling, lossless finite-state data compression, finite-state prediction, entropy rates, and automatic Kolmogorov complexity. The Schnorr-Stimm dichotomy theorem gave the first automata-theoretic characterization of normal sequences, which had been studied in analytic number theory since Borel defined them. This theorem implies that a sequence (or a real number having this sequence as its base-b expansion) is normal if and only if it has finite-state dimension 1. One of the most powerful classical tools for investigating normal numbers is the Weyl criterion, which characterizes normality in terms of exponential sums. Such sums are well studied objects with many connections to other aspects of analytic number theory, and this has made use of Weyl criterion especially fruitful. This raises the question whether Weyl criterion can be generalized from finite-state dimension 1 to arbitrary finite-state dimensions, thereby making it a quantitative tool for studying data compression, prediction, etc. This paper does exactly this. We extend the Weyl criterion from a characterization of sequences with finite-state dimension 1 to a criterion that characterizes every finite-state dimension. This turns out not to be a routine generalization of the original Weyl criterion. Even though exponential sums may diverge for non-normal numbers, finite-state dimension can be characterized in terms of the dimensions of the subsequence limits of the exponential sums. We demonstrate the utility of our criterion though examples.