TinyTim: A Family of Language Models for Divergent Generation
Christopher J. Agostino

TL;DR
TinyTim introduces a family of language models fine-tuned on Joyce's 'Finnegans Wake' to foster divergent, creative text generation, enabling conceptual reframing and breakthroughs beyond traditional convergent models.
Contribution
The paper presents a novel approach to creating divergent language models using fine-tuning on highly unconventional text, demonstrating increased lexical richness and resistance to factual convergence.
Findings
TinyTim-V1 exhibits over twenty times greater lexical richness than baselines.
TinyTim-V2 maintains divergence while resisting factual convergence.
The methodology enables specialized models for creative and reframing tasks.
Abstract
In the search for artificial general intelligence, model development and training has focused primarily on vast datasets of known problems and their accepted solutions. This process necessarily produces convergent systems which are fundamentally incapable of the conceptual reframing that is required for genuine creative breakthroughs. Inspired by the divergent cognitive processes that allow humans to make such creative leaps, our work introduces a family of language models, TinyTim, to serve as sources of divergent generation within broader systems. These models have been created by fine-tuning on the anti-parsimonious text of James Joyce's `Finnegans Wake'. Quantitative analysis of both an unsupervised fine-tuned model (TinyTim-V1) and a new instruction-tuned variant (TinyTim-V2) demonstrates a profound capacity for lexical invention; the foundational V1 model exhibits a Yule's K score…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
