Taming the Centaur(s) with LAPITHS: a framework for a theoretically grounded interpretation of AI performances

TL;DR

LAPITHS is a framework that critically assesses AI models' claims of human-like cognition, revealing that models like CENTAUR lack empirical and theoretical justification for such claims.

Contribution

It introduces LAPITHS, a novel interpretative framework that evaluates the cognitive plausibility of AI models using quantitative assessments and behavioral comparisons.

Findings

CENTAUR's claims of human-like cognition are not empirically justified.

Models similar to CENTAUR can produce comparable results without cognitive plausibility.

LAPITHS offers a principled method to critique AI claims of human-like intelligence.

Abstract

We introduce a framework called LAPITHS (Language model Analysis through Paradigm grounded Interpretations of Theses about Human likenesS) and use it to show that several major claims advanced by models such as CENTAUR, proposed as an artificial Unified Model of Cognition, are not theoretically or empirically justified. LAPITHS provides a principled reference point for counteracting the current behaviouristic tendency in AI research to interpret the human level performances of transformer based language models as evidence of human like underlying computation and, by extension, as signs of cognitive abilities. The novelty of LAPITHS lies in making explicit the arguments grounded in two quantitative assessments: (i) the Minimal Cognitive Grid, a theoretically motivated method for estimating the cognitive plausibility of artificial systems, and (ii) a behavioural comparison showing that results similar to those reported for CENTAUR like models can be reproduced by other systems that do not satisfy the structural constraints typically associated with cognitive plausibility, and whose outputs do not provide independent explanatory insight into human cognition.