`Generalization is hallucination' through the lens of tensor completions

TL;DR

This paper proposes using tensor completions and artifacts as a theoretical framework to understand hallucinations and generalizations in language models, offering new insights into their behavior.

Contribution

It introduces tensor completions and artifacts as a novel perspective for analyzing hallucinations in language models.

Findings

Tensor completions provide a useful framework for understanding hallucinations.

Artifacts in tensor completions relate to model generalization issues.

The approach offers new theoretical insights into language model behavior.

Abstract

In this short position paper, we introduce tensor completions and artifacts and make the case that they are a useful theoretical framework for understanding certain types of hallucinations and generalizations in language models.