Language processing in humans and computers

TL;DR

This paper compares human and machine language processing, highlighting how language models learn, hallucinate, and develop beliefs, emphasizing the parallels and differences between biological and artificial language understanding.

Contribution

It provides a high-level overview of language models and introduces a low-level model of learning machines, exploring their capacity for hallucinations and belief formation.

Findings

Language models can recognize hallucinations and learn to manage them.

Machines develop false beliefs and theories similar to humans.

A new low-level model of learning machines is proposed.

Abstract

Machine-learned language models have transformed everyday life: they steer us when we study, drive, manage money. They have the potential to transform our civilization. But they hallucinate. Their realities are virtual. This note provides a high-level overview of language models and outlines a low-level model of learning machines. It turns out that, after they become capable of recognizing hallucinations and dreaming safely, as humans tend to be, the language-learning machines proceed to generate broader systems of false beliefs and self-confirming theories, as humans tend to do.