The Imitation Game According To Turing
Sharon Temtsin (1), Diane Proudfoot (1), David Kaber (2), Christoph, Bartneck (1) ((1) The University of Canterbury, (2) Oregon State University)

TL;DR
This paper critically examines claims that large language models can pass the Turing Test by conducting a rigorous, standards-adherent imitation game, and finds that current models do not meet the criteria, challenging hype about their intelligence.
Contribution
The study provides a detailed, standards-based Turing Test on GPT-4-Turbo, revealing it cannot convincingly imitate humans, thus questioning recent claims of passing the Turing Test.
Findings
GPT-4-Turbo was identified by most participants as non-human.
None of the tested models convincingly passed the rigorous Turing Test.
Claims that LLMs can think are unsupported by strict Turing Test evaluations.
Abstract
The current cycle of hype and anxiety concerning the benefits and risks to human society of Artificial Intelligence is fuelled, not only by the increasing use of generative AI and other AI tools by the general public, but also by claims made on behalf of such technology by popularizers and scientists. In particular, recent studies have claimed that Large Language Models (LLMs) can pass the Turing Test-a goal for AI since the 1950s-and therefore can "think". Large-scale impacts on society have been predicted as a result. Upon detailed examination, however, none of these studies has faithfully applied Turing's original instructions. Consequently, we conducted a rigorous Turing Test with GPT-4-Turbo that adhered closely to Turing's instructions for a three-player imitation game. We followed established scientific standards where Turing's instructions were ambiguous or missing. For example,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsComputability, Logic, AI Algorithms
MethodsAttention Is All You Need · Softmax · Graph Self-Attention · RAdam · Hyperboloid Embeddings
