TinyTim: A Family of Language Models for Divergent Generation

TL;DR

TinyTim introduces a family of language models fine-tuned on Joyce's 'Finnegans Wake' to foster divergent, creative text generation, enabling conceptual reframing and breakthroughs beyond traditional convergent models.

Contribution

The paper presents a novel approach to creating divergent language models using fine-tuning on highly unconventional text, demonstrating increased lexical richness and resistance to factual convergence.

Findings

TinyTim-V1 exhibits over twenty times greater lexical richness than baselines.

TinyTim-V2 maintains divergence while resisting factual convergence.

The methodology enables specialized models for creative and reframing tasks.

Abstract

In the search for artificial general intelligence, model development and training has focused primarily on vast datasets of known problems and their accepted solutions. This process necessarily produces convergent systems which are fundamentally incapable of the conceptual reframing that is required for genuine creative breakthroughs. Inspired by the divergent cognitive processes that allow humans to make such creative leaps, our work introduces a family of language models, TinyTim, to serve as sources of divergent generation within broader systems. These models have been created by fine-tuning on the anti-parsimonious text of James Joyce's `Finnegans Wake'. Quantitative analysis of both an unsupervised fine-tuned model (TinyTim-V1) and a new instruction-tuned variant (TinyTim-V2) demonstrates a profound capacity for lexical invention; the foundational V1 model exhibits a Yule's K score for lexical richness over twenty times greater than that of convergent baselines. This trait is a stable property of the family, as the instruction-tuned V2 maintains a statistically distinct profile and resists factual convergence, sacrificing benchmark performance to preserve its core generative style. This work establishes a methodology for engineering specialized divergent models that, when paired with convergent systems, can reframe problems and force breakthroughs beyond the reach of statistical optimization alone.