Autoregressive Models for Knowledge Graph Generation

TL;DR

This paper introduces ARK, an autoregressive model for generating knowledge graphs by modeling semantic dependencies as sequences of triples, achieving high validity and enabling controlled, efficient graph generation.

Contribution

The paper presents ARK, a novel autoregressive approach for KG generation that captures complex dependencies without explicit rules and introduces SAIL for controlled graph synthesis.

Findings

Achieves 89.2% to 100% semantic validity on IntelliGraphs benchmark.

Recurrent architectures perform comparably to transformers with better efficiency.

Model capacity is more important than depth for effective KG generation.

Abstract

Knowledge Graph (KG) generation requires models to learn complex semantic dependencies between triples while maintaining domain validity constraints. Unlike link prediction, which scores triples independently, generative models must capture interdependencies across entire subgraphs to produce semantically coherent structures. We present ARK (Auto-Regressive Knowledge Graph Generation), a family of autoregressive models that generate KGs by treating graphs as sequences of (head, relation, tail) triples. ARK learns implicit semantic constraints directly from data, including type consistency, temporal validity, and relational patterns, without explicit rule supervision. On the IntelliGraphs benchmark, our models achieve 89.2% to 100.0% semantic validity across diverse datasets while generating novel graphs not seen during training. We also introduce SAIL, a variational extension of ARK that enables controlled generation through learned latent representations, supporting both unconditional sampling and conditional completion from partial graphs. Our analysis reveals that model capacity (hidden dimensionality >= 64) is more critical than architectural depth for KG generation, with recurrent architectures achieving comparable validity to transformer-based alternatives while offering substantial computational efficiency. These results demonstrate that autoregressive models provide an effective framework for KG generation, with practical applications in knowledge base completion and query answering.