Undecidability of $D_{<:}$ and Its Decidable Fragments

TL;DR

This paper proves that type checking and subtyping in a core calculus of Scala 3 are undecidable, but identifies and algorithms for two decidable fragments, with mechanized proofs.

Contribution

It establishes the undecidability of type checking in $D_{<:}$ and introduces two decidable subtyping fragments with algorithms and formal proofs.

Findings

Type checking in $D_{<:}$ is undecidable.

Two decidable subtyping fragments are identified.

Algorithms for the fragments are proven sound, complete, and terminating.

Abstract

Dependent Object Types (DOT) is a calculus with path dependent types, intersection types, and object self-references, which serves as the core calculus of Scala 3. Although the calculus has been proven sound, it remains open whether type checking in DOT is decidable. In this paper, we establish undecidability proofs of type checking and subtyping of $D_{<:}$, a syntactic subset of DOT. It turns out that even for $D_{<:}$, undecidability is surprisingly difficult to show, as evidenced by counterexamples for past attempts. To prove undecidability, we discover an equivalent definition of the $D_{<:}$ subtyping rules in normal form. Besides being easier to reason about, this definition makes the phenomenon of bad bounds explicit as a single inference rule. After removing this rule, we discover two decidable fragments of $D_{<:}$ subtyping and identify algorithms to decide them. We prove soundness and completeness of the algorithms with respect to the fragments, and we prove that the algorithms terminate. Our proofs are mechanized in a combination of Coq and Agda.