Measuring Understanding Through Discrete Compositional Knowledge Structures in Hierarchical Automata

TL;DR

This paper introduces a framework using hierarchical automata to produce discrete, inspectable signatures of understanding in artificial systems, enabling measurement of structural comprehension beyond probabilistic or neural representations.

Contribution

It presents a novel architecture combining finite state machines and higher-order automata to quantify and inspect understanding in artificial cognitive systems.

Findings

Five measurable signatures of understanding identified

Graph evolution tracking reveals structural knowledge and generalization

Framework distinguishes structural understanding from statistical correlation

Abstract

How do we measure genuine understanding in artificial cognitive systems? Current approaches face a measurement gap: probabilistic systems refine confidence gradually, practice-based systems compile knowledge through repeated execution, and neural systems distribute understanding across opaque embedding spaces. We propose that making understanding measurable requires architectures where understanding formation produces discrete, inspectable structural signatures. This paper presents hierarchical automata built from finite state machines representing patterns and higher-order automata representing compositions. Constrained inference constructs automata from single observations. Similarity detection clusters related automata, making concept robustness quantifiable. Graph memory makes compositional knowledge directly inspectable. Metacognitive mechanisms enable observable reconfiguration. We demonstrate understanding measurement in a simple geometric domain. Graph evolution tracking reveals five measurable signatures: immediate representation formation, structural knowledge, generalization capacity, compositional awareness, and metacognitive access. These measurements distinguish structural understanding from statistical correlation. Our contribution is a framework for making understanding measurable through discrete compositional knowledge structures. This measurement capability complements perceptual learning in neural systems and task execution in neurosymbolic architectures.