(Un)biased data and spin glasses reveal clustering for Turing phase transitions within human-transformer interactions

TL;DR

This paper investigates phase transitions in large language models through statistical, information retrieval, and spin glass models, revealing clustering phenomena in human-AI interactions and proposing a new theoretical framework for AI analysis.

Contribution

It introduces a novel interdisciplinary approach combining spin glass theory, statistical analysis, and information retrieval to study AI-human interaction phase transitions.

Findings

Identified temperature-induced phase transitions in LLMs.

Demonstrated clustering in human-AI interaction data.

Aligned results with existing transformer and metastability literature.

Abstract

This paper studies a Large Language Model's ability to exhibit intelligence equivalent to that of a human by analyzing temperature-induced phase transitions, abrupt changes in the macroscopic behavior of a system, in the Turing test. We utilize three approaches: statistical analysis and bias quantification of a human evaluation survey, information retrieval from real human-written versus AI-generated text data using cosine similarity as a comparison metric, and mathematical spin glass model and simulation. We collect text data in the case study of Flitzing, a tradition of emailing poem-like romantic invitations at Dartmouth College because of its richness in information. Across the three approaches, we obtain consistency in phase transition and clustering results, which also align with literature on the mathematics of transformers and metastability. Our work inspires utilizing spin glass theory for the mathematical foundations of artificial intelligence, especially under environmental stochasticity from human interactions, with justification from real data.