Undecidability of the submonoid membership problem for a sufficiently large finite direct power of the Heisenberg group

TL;DR

This paper proves that for large enough direct powers of the Heisenberg group, the submonoid membership problem becomes undecidable, answering open questions about nilpotent groups and their algorithmic properties.

Contribution

It demonstrates the existence of finitely generated submonoids with undecidable membership problems in large direct powers of the Heisenberg group, extending to free nilpotent groups.

Findings

Existence of finitely generated submonoids with undecidable membership in large Heisenberg powers

Answers to open questions by Lohrey, Steinberg, Colcombet, Ouaknine, Semukhin, and Worrell

Undecidability derived from Hilbert's 10th problem and Diophantine equations in nilpotent groups

Abstract

The submonoid membership problem for a finitely generated group $G$ is the decision problem, where for a given finitely generated submonoid $M$ of $G$ and a group element $g$ it is asked whether $g \in M$. In this paper, we prove that for a sufficiently large direct power $\mathbb{H}^n$ of the Heisenberg group $\mathbb{H}$, there exists a finitely generated submonoid $M$ whose membership problem is algorithmically unsolvable. Thus, an answer is given to the question of M. Lohrey and B. Steinberg about the existence of a finitely generated nilpotent group with an unsolvable submonoid membership problem. It also answers the question of T. Colcombet, J. Ouaknine, P. Semukhin and J. Worrell about the existence of such a group in the class of direct powers of the Heisenberg group. This result implies the existence of a similar submonoid in any free nilpotent group $N_{k,c}$ of sufficiently large rank $k$ of the class $c\geq 2$. The proofs are based on the undecidability of Hilbert's 10th problem and interpretation of Diophantine equations in nilpotent groups.