On the Borel Inseparability of Game Tree Languages

TL;DR

This paper proves that certain game tree languages at the first non-trivial level are not Borel separable, demonstrating a fundamental limitation in their definability within the Borel hierarchy.

Contribution

It establishes the Borel inseparability of specific game tree languages at the first non-trivial level, resolving an open case in the hierarchy of automata and fixed-point logic.

Findings

Game tree languages at the first non-trivial level are co-analytic and Borel inseparable.

The result completes the understanding of separation properties in the automata hierarchy.

It confirms that these languages cannot be separated by any Borel set, including weakly definable sets.

Abstract

The game tree languages can be viewed as an automata-theoretic counterpart of parity games on graphs. They witness the strictness of the index hierarchy of alternating tree automata, as well as the fixed-point hierarchy over binary trees. We consider a game tree language of the first non-trivial level, where Eve can force that 0 repeats from some moment on, and its dual, where Adam can force that 1 repeats from some moment on. Both these sets (which amount to one up to an obvious renaming) are complete in the class of co-analytic sets. We show that they cannot be separated by any Borel set, hence {\em a fortiori} by any weakly definable set of trees. This settles a case left open by L.Santocanale and A.Arnold, who have thoroughly investigated the separation property within the $\mu $-calculus and the automata index hierarchies. They showed that separability fails in general for non-deterministic automata of type $\Sigma^{\mu}_{n} $, starting from level $n=3$, while our result settles the missing case $n=2$.