Evidence of interrelated cognitive-like capabilities in large language models: Indications of artificial general intelligence or achievement?

TL;DR

This study investigates whether large language models exhibit interrelated cognitive-like abilities similar to human intelligence, finding evidence of a general ability factor and group-level factors across diverse tests, with larger models showing enhanced capabilities.

Contribution

It provides empirical evidence of a general intelligence factor in LLMs and explores how model size relates to cognitive-like abilities, a novel investigation into AI generality.

Findings

Strong positive manifold among LLM test scores

Identification of a general ability factor in LLMs

Larger models tend to have higher general ability scores

Abstract

Large language models (LLMs) are advanced artificial intelligence (AI) systems that can perform a variety of tasks commonly found in human intelligence tests, such as defining words, performing calculations, and engaging in verbal reasoning. There are also substantial individual differences in LLM capacities. Given the consistent observation of a positive manifold and general intelligence factor in human samples, along with group-level factors (e.g., crystallized intelligence), we hypothesized that LLM test scores may also exhibit positive intercorrelations, which could potentially give rise to an artificial general ability (AGA) factor and one or more group-level factors. Based on a sample of 591 LLMs and scores from 12 tests aligned with fluid reasoning (Gf), domain-specific knowledge (Gkn), reading/writing (Grw), and quantitative knowledge (Gq), we found strong empirical evidence for a positive manifold and a general factor of ability. Additionally, we identified a combined Gkn/Grw group-level factor. Finally, the number of LLM parameters correlated positively with both general factor of ability and Gkn/Grw factor scores, although the effects showed diminishing returns. We interpreted our results to suggest that LLMs, like human cognitive abilities, may share a common underlying efficiency in processing information and solving problems, though whether LLMs manifest primarily achievement/expertise rather than intelligence remains to be determined. Finally, while models with greater numbers of parameters exhibit greater general cognitive-like abilities, akin to the connection between greater neuronal density and human general intelligence, other characteristics must also be involved.