Large Language Models and Emergence: A Complex Systems Perspective

TL;DR

This paper explores the emergence of complex properties in large language models through a complex systems lens, examining whether their capabilities qualify as emergent intelligence and how this relates to complexity science concepts.

Contribution

It provides a comprehensive review of emergence in the context of large language models and discusses whether LLMs exhibit true emergent intelligence from a complex systems perspective.

Findings

Reviewed approaches to quantifying emergence in LLMs

Analyzed the relationship between emergence and intelligence in LLMs

Discussed the implications of emergence for understanding LLM capabilities

Abstract

Emergence is a concept in complexity science that describes how many-body systems manifest novel higher-level properties, properties that can be described by replacing high-dimensional mechanisms with lower-dimensional effective variables and theories. This is captured by the idea "more is different". Intelligence is a consummate emergent property manifesting increasingly efficient -- cheaper and faster -- uses of emergent capabilities to solve problems. This is captured by the idea "less is more". In this paper, we first examine claims that Large Language Models exhibit emergent capabilities, reviewing several approaches to quantifying emergence, and secondly ask whether LLMs possess emergent intelligence.