Absence of Phase Transition in Random Language Model

TL;DR

This paper demonstrates that the Random Language Model, based on probabilistic context-free grammar, does not exhibit a phase transition, challenging previous assumptions and suggesting the need for more complex models like context-sensitive grammars.

Contribution

The study provides a theoretical analysis showing the absence of phase transition in the Random Language Model by reducing the problem to eigenvector analysis of transition matrices.

Findings

No phase transition occurs in the Random Language Model.

Eigenvector analysis reveals the absence of order emergence.

More complex models may be needed for phase transitions.

Abstract

The Random Language Model, proposed as a simple model of human languages, is defined by the averaged model of a probabilistic context-free grammar. This grammar expresses the process of sentence generation as a tree graph with nodes having symbols as variables. Previous studies proposed that a phase transition, which can be considered to represent the emergence of order in language, occurs in the random language model. We discuss theoretically that the analysis of the "order parameter" introduced in previous studies can be reduced to solving the maximum eigenvector of the transition probability matrix determined by a grammar. This helps analyze the distribution of a quantity determining the behavior of the "order parameter" and reveals that no phase transition occurs. Our results suggest the need to study a more complex model such as a probabilistic context-sensitive grammar, in order for phase transitions to occur.