Structured Relational Representations

TL;DR

This paper proposes a formal framework for structured relational invariant representations in learning systems, emphasizing the importance of partitions in an abstract knowledge space for stable and transferable knowledge.

Contribution

It introduces a novel formalization of invariant representations as partitions defined by relational paths within an abstract knowledge space, grounded in relational algebra.

Findings

Invariant partitions serve as the core of structured knowledge representations.

Relational algebra provides a formal foundation for these invariant structures.

Partitions enable stable, transferable knowledge in learning systems.

Abstract

Invariant representations are core to representation learning, yet a central challenge remains: uncovering invariants that are stable and transferable without suppressing task-relevant signals. This raises fundamental questions, requiring further inquiry, about the appropriate level of abstraction at which such invariants should be defined and which aspects of a system they should characterize. Interpretation of the environment relies on abstract knowledge structures to make sense of the current state, which leads to interactions, essential drivers of learning and knowledge acquisition. Interpretation operates at the level of higher-order relational knowledge; hence, we propose that invariant structures must be where knowledge resides, specifically as partitions defined by the closure of relational paths within an abstract knowledge space. These partitions serve as the core invariant representations, forming the structural substrate where knowledge is stored and learning occurs. On the other hand, inter-partition connectors enable the deployment of these knowledge partitions encoding task-relevant transitions. Thus, invariant partitions provide the foundational primitives of structured representation. We formalize the computational foundations for structured relational representations of the invariant partitions based on closed semiring, a relational algebraic structure.