Circuit Complexity of Hierarchical Knowledge Tracing and Implications for Log-Precision Transformers

TL;DR

This paper explores the computational complexity of hierarchical knowledge tracing using circuit complexity theory, revealing theoretical limits and proposing structure-aware training methods for transformers on deep concept hierarchies.

Contribution

It formalizes the complexity of prerequisite propagation in transformers, establishes new circuit complexity bounds, and demonstrates the effectiveness of auxiliary supervision in structure-aware knowledge tracing.

Findings

Recursive-majority propagation is in NC^1, but separating it from TC^0 remains open.

Transformer encoders converge to shortcuts on recursive-majority trees without explicit structure.

Auxiliary supervision on subtrees improves accuracy and induces structure-dependent computation.

Abstract

Knowledge tracing models mastery over interconnected concepts, often organized by prerequisites. We analyze hierarchical prerequisite propagation through a circuit-complexity lens to clarify what is provable about transformer-style computation on deep concept hierarchies. Using recent results that log-precision transformers lie in logspace-uniform $\mathsf{TC}^0$, we formalize prerequisite-tree tasks including recursive-majority mastery propagation. Unconditionally, recursive-majority propagation lies in $\mathsf{NC}^1$ via $O(\log n)$-depth bounded-fanin circuits, while separating it from uniform $\mathsf{TC}^0$ would require major progress on open lower bounds. Under a monotonicity restriction, we obtain an unconditional barrier: alternating ALL/ANY prerequisite trees yield a strict depth hierarchy for \emph{monotone} threshold circuits. Empirically, transformer encoders trained on recursive-majority trees converge to permutation-invariant shortcuts; explicit structure alone does not prevent this, but auxiliary supervision on intermediate subtrees elicits structure-dependent computation and achieves near-perfect accuracy at depths 3--4. These findings motivate structure-aware objectives and iterative mechanisms for prerequisite-sensitive knowledge tracing on deep hierarchies.